Test Run: Apple Watch Vs Garmin Forerunner 220
Published on
06 Mar 2016
Written by
Andrew Sargent
Director. Titled Sports Physiotherapist.
Call us on: (03) 9975 4133
Wearable technology is the latest emerging craze amongst fitness fans. And with good reason. Technology has enhanced every aspect of our lives – from the way we communicate and entertain ourselves to the way we work. Now, let’s see how these innovations really impact upon our workout.
Latest Update: 06/03/2016 – Click to see the updated results!
Experiment with exercise, improve your fitness.
Elite athletes have always enjoyed a scientific approach to their exercise programs. This tactic plays a large role in how they become such high-powered and finely-tuned machines.
Now, thanks to wearable technologies, you and I have access to the information required for the same kind of approach. The benefits are huge.
Wearables can provide amazing insights into the way you personally function as an athlete, which enables you to gauge, track and hopefully improve performance over time. Along with standard metrics like speed and distance covered, we can now record specific data such as heart rate information, stride analysis and even lactate inflection points. It’s a runner’s dream.
But before we get too excited about the prospect of getting on the track and tracking progress in exhaustive detail, let’s do some research.
The “test run”.
I’ve decided to put two popular devices through their paces – while they do the same to me.
Simply put, over the course of this month, I’ll be conducting a runner’s review of the Apple Watch and the Garmin Forerunner 220, to see how accurate these devices are when tested against one another. I’m using the standard fitness app for the Apple Watch, being ‘Workout’, from the ‘Activity’ section. In regard to GPS tracking and data carriers, I’m using Telstra as a supplier.
Apple Watch Vs Garmin Forerunner 220
Here’s how it will work:
- I’ll wear the Garmin watch on my right wrist. The Apple Watch is on my left.
- I conducted an initial “calibration run” with an iPhone and both watches for 30 minutes, in an open area with a good GPS signal. This was to calibrate and set-up the Apple Watch (as per the instruction manual – see the results below).
- I will now run home from work two times per week throughout February, wearing both watches.
- I will carry the iPhone for the first week (to further improve the accuracy of the Apple Watch and to enhance the calibration, as per the instruction manual).
- I won’t carry the iPhone for the next 2 weeks (four runs), and simply wear both watches. This will test Apple’s claims of “learning your stride length” and the ability to stay accurate without a phone for GPS.
- I will carry the iPhone again for the final week (two runs).
- I will compare the data from the Apple Watch with the data from the Garmin.
- I will compare the data from the Apple Watch when carrying the iPhone with the data recorded without the iPhone.
- As a final test of accuracy, I’ll run a set and known distance at the end of February – ten laps of an outdoor 400m running track for a distance of 4km. I’ll do this two times. Once with the iPhone and both watches, and once with no iPhone, just the two watches. I’ll check and analyse the results.
- Decide which watch I’m going to keep wearing!
The route.
Mapometer identified the distance of my run as 7.6km:
Results and analysis.
Monday, February 1st, 2016 – The “calibration run”
I ran home from work wearing the Garmin Forerunner 220, the Apple Watch, and while carrying my iPhone (in order to provide GPS for the Apple Watch). The weather was fantastic, with perfect conditions for both the GPS and the run in general. There wasn’t a cloud in the sky and the temperature was a balmy 23 degrees Celsius.
Here are the initial results:
Apple Watch
- Distance Recorded: 5.27km
- Time Recorded: 34 minutes, 14 seconds
- Average Pace: 6’29” per kilometre
Garmin Forerunner 220
- Distance Recorded: 6.73km
- Time Recorded: 34 minutes, 10 seconds
- Average Pace: 5’05” per kilometre
Despite all that good weather we’ve got some foggy results. Clearly not the best start in terms of accuracy!
There is a huge discrepancy between the devices initially, with a distance differential of 1.46km, and both fall short of the actual distance covered (assuming Mapometer is correct, of course).
Mapometer recorded my run as being 7.6km. Since the time was consistent at around 34 minutes and 12 seconds, this would make the average speed 4’30” per km.
I’ve been short-changed in the time stakes! Or, maybe, if the devices provide a low baseline reading, I’ll feel superhuman when my actual time is revealed later on. I’ve improved! And it’s all thanks to the watch!
In reality, I suspect this is probably due to the fact that the Apple Watch was calibrating itself for the first part of the run rather than recording any useful information (unfortunately, this was unavoidable).
Apple states that the watch requires an initial 30 minutes of running with the iPhone for optimum performance (i.e. this run), so it’ll be interesting to see whether the gaps in data shrink over the coming runs. Especially given Apple’s claims that the more frequently you run with an iPhone GPS, the more accurate your results. The device supposedly “studies” your running – brings a new meaning to the saying “watch and learn”, doesn’t it?
Friday, February 5th, 2016 – Run #1 – the first official test
Now that the Apple Watch is calibrated, we can get down to the real deal. So here goes: I ran my standard route from the CBD again. Once again, the weather was absolutely perfect, with blue skies above and temperatures hovering in the mid-twenties. So how did the results stack up?
Apple Watch
- Distance Recorded: 6.93km
- Time Recorded: 33 minutes, 49 seconds
- Average Pace: 4’52” per kilometre
Garmin Forerunner 220
- Distance Recorded: 6.92km
- Time Recorded: 33 minutes 49 seconds
- Average Pace 4’53” per kilometre
Now, that’s more along the lines of what I was expecting.
The data recorded by both watches is crazily similar, although they have both cut three quarters of a kilometer off the total run distance.
That discrepancy could be due to the fact that part of the route goes through the Melbourne CBD, which is chock full of tall buildings. Huge enough to disrupt the GPS signal that is mapping the run? Potentially. I guess we’ll find out as we continue this exercise experiment.
So now we’re off and racing. If you’re interested in following my progress to see how things tick along with the test-runs, be sure to check back for progress updates and further analysis of both products.
In the meantime, that’s a tick for both of these watches as far as I’m concerned. Clearly comparable results, and impressively similar time recording (and, not too a bad time for the run overall, if I do say so myself).
Bring on the rest of the running!
Monday, February 8th, 2016 – Run #2 – things are getting interesting!
Another interesting run. 23 degrees, and not a cloud in the sky. Here’re the results of today’s run:
Apple Watch
- Distance Recorded: 5.49km
- Time Recorded: 33 minutes, 33 seconds
- Average Pace: 6’06” per kilometre
Garmin Forerunner 220
- Distance Recorded: 7.45km
- Time Recorded: 33 minutes 32 seconds
- Average Pace 4’30” per kilometre
Not really sure what to make of this. It’s almost like the Garmin is learning, but the Apple Watch is not!
Looking at the splits (time per kilometre) on the Apple Watch they read:
4’51”
5’26”
5’23”
5’22”
5’28”
14’47”
Clearly an issue with that last kilometre, and it perhaps debunks my ‘city GPS theory’ that I had on Friday as—if this held true—I would have expected the splits to be more accurate as I got closer to home (and therefore further away from city buildings).
I typically run at 4’30” to 4’45” pace, and although I tend to go a bit slower up Mount Alexander road (the 4-5km mark with the splits), the Apple results are consistently off at well over 5 mins per kilometre.
My other issue—and hopefully all you helpful people on social media can assist me with this?—is that the distance on the Apple Watch is not updating in real time.
Today for example I started running and the Apple Watch continued to read 0 distance, then it suddenly jumped to .27km, where it remained as the time went up. As a result, the average pace blew out to over 10 mins per km. Then the distance jumped to 2.7km (at which time the Garmin was already displaying just over 3km). There was only one more distance change for the Apple Watch prior to finishing.
I’m not really sure why this is happening, does anyone have a theory? Would love to hear your thoughts on Facebook or Twitter!
The next run I do is supposed to be without the iPhone. It will certainly be very interesting to see what the Apple Watch makes of this! Maybe it will be more accurate without the GPS!
Friday, February 12th, 2016 – Run #3 – Apple, what’s going on?!
So, run 3 has been completed. Another beautiful day in Melbourne, sunny and 24 degrees.
Despite the Apple watch turning in pretty average data to date, I followed my original plan and ran without the iPhone. This is where the Apple watch is supposed to use previously recorded data to predict my run distance (without the aid of a GPS). Unfortunately it looks like my Apple watch did exactly this, and used the previously recorded inaccurate data to make another inaccurate prediction! I fact, if you have a look at the graph above, you’ll see the Apple Watch is actually getting worse!
Meanwhile the Garmin continued to be fairly close:
Apple Watch
- Distance Recorded: 4.49km
- Time Recorded: 34 minutes, 25 seconds
- Average Pace: 7’39” per kilometre
Garmin Forerunner 220
- Distance Recorded: 7.2km
- Time Recorded: 34 minutes 26 seconds
- Average Pace 4’45” per kilometre
The inaccuracies of the Apple watch have led me to where all good solutions are found….blogs! It appears I am not alone with my experiences to date. The advice from Apple, and other users, is to ‘un-pair my watch from my iPhone, ensure software for my iPhone and Apple watch is up to date, then re-pair my watch to my phone and finally complete the calibration process again’.
This looks like a job for Saturday.
Check back next week for round 2 of re-calibrated Apple watch Vs Garmin!
Tuesday, February 16th, 2016 – Run #4 – The Apple Watch sorts itself out
After the problems experienced with the Apple Watch in previous runs, I decided to go through the process of re-pairing the Apple Watch with my iPhone and then recalibrating the watch via a 30min run in an open space. I did this at the weekend, prior to run #4.
It looks like this may have rectified the previous inaccuracies, with the data for run #4 looking much closer to the actual distance covered. Here are the results:
Apple Watch
- Distance Recorded: 7.49km
- Time Recorded: 34 minutes, 30 seconds
- Average Pace: 4’36” per kilometre
The splits on the Apple Watch read:
5’05”
4’31”
4’24”
4’29”
4’30”
4’51”
4’31”
4’10”
Garmin Forerunner 220
- Distance Recorded: 7.12km
- Time Recorded: 34 minutes 30 seconds
- Average Pace 4’51” per kilometre
For the first time in this ongoing test, the Apple Watch has provided what I would consider to be an acceptably-accurate result, recording an overall distance covered of 7.49km, which is very close to the actual distance of 7.6km.
However, I was carrying my iPhone… and I’m still sceptical as to whether the Apple Watch can provide an accurate result without me having to carry my iPhone.
Thursday’s run will be a repeat of tonight’s run, with both watches and the iPhone in situ. I’m hoping for some more consistency, but we’ll have to wait and see.
Stay tuned!
Friday, February 19th, 2016 – Run #5 – Better, but not perfect
Another run completed… here are the results:
Apple Watch
- Distance Recorded: 7.27km
- Time Recorded: 34 minutes, 45 seconds
- Average Pace: 4’46” per kilometre
Garmin Forerunner 220
- Distance Recorded: 7.44km
- Time Recorded: 34 minutes 43 seconds
- Average Pace 4’40” per kilometre
The second run post re-calibration saw the Garmin out perform the Apple watch. Now that the Apple Watch distance is updating in real time I’m able to check the splits as I go. For some reason kilometre 2 of my run saw the Apple Watch lose 200 metres on the Garmin; a distance it failed to catch up. It would have been another fairly accurate result for the Apple Watch if it hadn’t been for that minor slip.
Next week I’m going back to running without the iPhone to test Apple’s claims that a calibrated watch can use its accelerometer to predict distance. Let’s see how it goes!
Happy running!
Monday, February 22nd, 2016 – Run #6 – No iPhone, and the Apple Watch struggles
Here are the results from today’s run:
Apple Watch
- Distance Recorded: 6.34km
- Time Recorded: 33 minutes, 35 seconds
- Average Pace: 5’17” per kilometre
Garmin Forerunner 220
- Distance Recorded: 7.5km
- Time Recorded: 33 minutes 35 seconds
- Average Pace 4’29” per kilometre
The iPhone was turned off during this run to test Apple’s claim that it works out your pace during the calibration runs. Unfortunately, as per run #3, the results from the Apple Watch are highly inaccurate, recording a total distance of 6.34 kilometres, well over a kilometre short of the actual distance.
The Garmin Forerunner put in a very respectable result of 7.5 kilometres, its best yet.
Thursday, February 25th, 2016 – Run #7 – Apple Watch: misses the mark again!
Here are the results from today’s run:
Apple Watch
- Distance Recorded: 6.4km
- Time Recorded: 34 minutes, 25 seconds
- Average Pace: 5’22” per kilometre
Garmin Forerunner 220
- Distance Recorded: 7.27km
- Time Recorded: 34 minutes 23 seconds
- Average Pace 4’44” per kilometre
Again, the Apple Watch misses the mark by quite some distance. This is despite having over an hour of pretty accurate data for the Apple Watch to utilise (for estimation purposes) following the recalibration. The Garmin records a good result, yet again.
I think there’s a fairly common theme here. For recording your running performance, the Apple Watch is only suitable if you’re willing to run with your iPhone.
Next week I will complete a track run in order to test the two watches over a fixed distance (10 laps of a 400m athletics track).
Repeating the test without the iPhone immediately afterwards should provide some further insight into the accuracy of the Apple watch sans iPhone.
Check back in next week!
Sunday, March 6th, 2016 – Run #8 – The final accuracy test.
I’ve called the whole series a ‘test run’, but this run was a real test – in the truest sense of the word. By running 10 laps of a conventional 400m running track, I had a standardised distance of 4km to test each device, in varying circumstances. Put plainly, in this instance we’ve got a pretty good idea of the data we should receive. If the information is a long way off, the watch has come up short.
It’s worth noting that the distance of each lap is always slightly longer than the measured 400m because it’s not possible to exactly follow the inside line of the track, where the precise measurement is recorded.
With that cleared up, let’s get down to what happened.
Here are the stats:
Run 8a – 10 x 400m laps with Apple watch and iPhone, and Garmin.
Apple Watch (with iPhone)
- Distance Recorded: 4.25km
- Time Recorded: 17 minutes, 41 seconds
- Average Pace: 4’09” per kilometre
Garmin Forerunner 220
- Distance Recorded: 4.16km
- Time Recorded: 17 minutes, 41 seconds
- Average Pace: 4’15” per kilometre
Run 8b – 10 x 400m laps with Apple watch but NO iPhone, and Garmin.
Apple Watch (without iPhone)
- Distance Recorded: 3.50km
- Time Recorded: 17 minutes, 44 seconds
- Average Pace: 5’03” per kilometre
Garmin Forerunner 220
- Distance Recorded: 4.16km
- Time Recorded: 17 minutes, 44 seconds
- Average Page: 4’16” per kilometre
Once again, the Apple watch showcases a tendency to disappoint with the data it offers up. At this point, it’s simply not accurate without the iPhone in tow. On this occasion it only registered 84% of Garmin’s distance. As I mentioned earlier, the actual distance covered on each run is likely to be slightly higher than 4km. The 4.16km distance that the Garmin device recorded is probably very close to the actual distance that I ran.
This is the first run that I completed away from the Melbourne CBD, and therefore away from the skyscrapers and tall buildings that might impact negatively on a GPS. It’s also the first time in this entire experiment that both the Apple and the Garmin have been exactly on (or just slightly over) the actual measurement for distance.
It’s entirely possible that the GPS is affected for the first few kilometres of any run through the city centre. So if you’re leaving work to enjoy a quick lap of the tan before heading home, take your data with a grain of salt. You could be selling yourself short. Alternatively, allow a sufficient chunk of time for the GPS to find a signal before you head out on a run, or if your device has been switched off throughout the day.
Well, that signals the finish line for this initial round of experimental ‘test runs’. As with any sporting event though, there’s going to be some ongoing discussion, endless analysis and in-depth afters.
I’ll be writing up a detailed outline of the whole experiment. I’ll compare data across the whole journey, and offer up my review of each product, some insight into my experience throughout the tests, and a final verdict.
Stay tuned and check back shortly for the analysis!
About the Author
Andrew Sargent — Director. Titled Sports Physiotherapist.
Andrew is most passionate about working with clients who have a sport, exercise or fitness goal. Helping these clients recover from injury and then return to a healthy, active lifestyle is such a buzz.
