HDOP, PDOP, RMS and FIX/FLOAT: which GNSS quality indicator should you trust?

RTK fix vs float: the status that overrides everything else

What FIX and FLOAT actually mean

A carrier-phase measurement is precise but ambiguous: it contains an unknown whole number of wavelengths. Resolving that number is what NovAtel calls the key to unlocking the highest-accuracy GNSS positions (retrieved September 2026). Until it is, the receiver estimates the ambiguities as real numbers, and NGS puts it plainly: that decimal cycle count "is said to be the float solution, one that still has not yet forced the number of whole cycles to take an integer value".

The two states are not two grades of one answer. Your rover announces which is running: u-blox documents quality indicator 4 as RTK fixed and 5 as RTK float in the GGA sentence (retrieved September 2026), alongside the satellite count and the HDOP GPS receivers compute for that epoch.

Vendors quantify float differently. Septentrio says it converges "from several decimeters to several centimeters" over several minutes; Emlid puts it at "a few centimeters to about 1 meter". The shorthand that float means decimetre-level sits between the two and is not a specification either company publishes.

Why a good geometry number means little under float

Under float the ambiguities are unresolved, so the position is a converging estimate. The accuracy figure still describes something real, but NGS is careful about what: a data collector reports internal repeatability, and warns the display can show "false precision" when multipath is present. Tight numbers under float describe a receiver agreeing with itself, not with the ground, and geometry cannot settle it either way.

RMS: the statistic your software is quietly rounding

RMS and the confidence-level problem (CEP, HRMS, 2DRMS, R95)

Most arguments about GNSS accuracy are really arguments about confidence levels. The US Army Corps of Engineers is blunt: a "3-meter" accuracy statistic "is meaningless unless it is identified as being either 1-D, 2-D, or 3-D, along with the applicable probability or confidence level". A 2025 FIG paper prints the horizontal conversions (retrieved September 2026).

Two traps live in that table. Horizontal one sigma is 39.35 percent, not 68 percent; the 68.3 percent figure belongs to the one-dimensional vertical case, and Frank van Diggelen listed the confusion as a named misconception, true "for only 1-D Gaussian distributions" (retrieved September 2026). And 2DRMS is not a flat 95 percent: it encloses 98 percent for a circular distribution and drifts toward 95 percent as the error ellipse elongates, which is what weak horizontal geometry does.

Why "1 cm" from two different receivers is not the same claim

Compare two datasheets. The u-blox ZED-F9P-04B quotes RTK horizontal accuracy of 0.01 m + 1 ppm CEP, vertical as Median (retrieved September 2026): both 50 percent statistics, so half your points fall outside. Trimble quotes its R12i RTK performance as RMS instead. The digits look alike; the promises do not. Van Diggelen's 2007 update adds the rule for unlabelled specifications: an accuracy quoted with no metric "is usually CEP".

One caveat applies here specifically. Van Diggelen notes differential and RTK errors over short periods are dominated by multipath and "distinctly non-Gaussian". Survey occupations are short, so these factors are least reliable exactly where this article is aimed: use them to compare specifications, not to convert your own observations.

HDOP and PDOP: what satellite geometry is telling you

HDOP vs PDOP vs VDOP, and PDOP² = HDOP² + VDOP²

Dilution of precision is a scaling factor drawn from satellite geometry and nothing else. Richard Langley defines it through the square root of the trace of the solution covariance matrix, whose elements "are a function of the receiver-satellite geometry only". The values are dimensionless, as u-blox states in its protocol specification, and lower is better.

The family splits by which part of that covariance you sum: horizontal takes the east and north terms, vertical the up term, and the position figure all three. That is why NOAA's National Geodetic Survey can print the identity as "PDOP² = HDOP² + VDOP²". Add the clock term for GDOP, take it alone for TDOP. The identity is exact only within one solution covariance in one local frame, so figures from different receivers will not reproduce it.

The relationship to error is equally conditional. The GPS Standard Positioning Service Performance Standard gives Accuracy = UERE x DOP, then calls it "a simple approximation" valid only when all pseudorange errors are zero mean, normally distributed and share the same UERE. Its outputs are RMS quantities. And it scales precision rather than accuracy: NGS words it "lower DOP values should indicate better precision".

What counts as a good value, and why the thresholds do not agree

There is no single answer, and the sources are not being careless. They gate different things.

The spread is not noise. NGS assigns a different ceiling to each accuracy class, the clearest published demonstration that the limit follows from the work. The 6 appearing everywhere else has other parentage: in the SPS Performance Standard it is a constellation-availability commitment covering 98 percent of the global average over a sidereal day, which says nothing about your point. Trimble brackets the disagreement by itself, its glossary calling anything above 3 not good while its field software ships a mask of 6, and the chip-level default is looser again because there the number is a validity flag, not a quality gate. If you have heard that 6 is the standard mask and 4 is for precision work, the first half describes Trimble Access and an old ArcPad rule of thumb; the second is field-training folklore, not a documented default anywhere we could find.

Why these four numbers can disagree with each other

Most guides stop at the warning not to conflate geometry with error. The more useful question, and the one that decides your GNSS accuracy in practice, is what a specific contradiction tells you.

Good DOP, bad RMS: what it means

A 2024 forest study is the cleanest published example. Under canopy with a four-constellation survey-grade receiver, the authors recorded about 13 visible satellites on average and a PDOP GPS surveyors would accept without hesitation, under 2. They still obtained no fixed solutions at all across 2,670 epochs, leaving the receiver in place for one to two hours at some points (retrieved September 2026). Horizontal accuracy came out at 2.03 m RMSE.

Nothing was wrong with the sky map. The signals were attenuated and contaminated by multipath, and the receiver could not hold carrier phase lock long enough to resolve ambiguities. Geometry describes where the satellites are, not whether their signals arrive intact. Teunissen and colleagues give the formal version: the position figure "works well for code-based positioning" but needs "great care" in RTK, because a small position figure can coincide with a large ambiguity figure, in which case ambiguity resolution will not be possible.

FIX solution, high PDOP: what it means

The reverse case is real too, and the same work notes a large position figure with a small ambiguity figure is not necessarily poor performance. But a fixed solution is not automatically correct. NovAtel documents the failure mode and ships a command for it: normally a fixed-integer solution is very accurate, "however, in some rarely-occurring situations, even a fixed-integer solution can become inaccurate; for example, if the DOP is high due to satellites not being visible". Its RTKINTEGERCRITERIA command reports such a solution as float instead. u-blox makes the same admission from the other side with a conservative ambiguity fix mode, trading a slower fix for "near absolute certainty when RTK is achieved" in urban surveying. A fast fix in a hard environment deserves suspicion.

A field decision framework

Work the numbers in order, and let each qualify the next.

1. Solution status. Fixed or float, read as words rather than a numeric code. Under float, stop and fix the cause before recording anything you intend to keep.
2. Correction age. A fixed solution on stale corrections is a fixed solution drifting. Our documentation puts full performance under one second of latency, with more float above one to three seconds.
3. Accuracy estimate, with its confidence basis. Establish whether your software shows CEP, one sigma or 2DRMS before comparing it to a specification.
4. Geometry. Read it last, as an explanation. Good geometry does not validate the numbers above it; poor geometry tells you which of them to distrust.

A worked case: a fixed solution, a correction age of two seconds, a horizontal figure of 0.008 m, and a PDOP GPS field software would flag as marginal at 5.4. Step one passes. Next, step two is the first thing to chase. Step three needs a check, because if that 0.008 m is CEP then the 95 percent figure is nearer 0.017 m, which may still meet your tolerance but is not what the screen implied. Finally, step four reads as a warning rather than a verdict, and argues for reoccupying the point.

Reverse it and the reading changes completely. A float solution with excellent geometry is not a good measurement with a small problem; it is a measurement whose ambiguities were never resolved, which is why what centimetre accuracy means in the field depends on solution status before anything else.

What degrades them, and why they rarely fall together

The intuition that a hard environment pushes all four indicators the same way is worth abandoning. The forest study kept good geometry and lost the fix entirely, because signal quality and geometry degrade through different mechanisms.

Satellite count sets the floor. u-blox states a single-constellation receiver needs at least 6 satellites with continuous phase lock above the elevation mask to attempt a fixed solution, 7 for GPS plus GLONASS, and 8 once BeiDou joins, so adding constellations helps most where the sky is restricted. Canopy then attacks the signal rather than the geometry: a ZED-F9P in Slovakia lost 4 to 5 dB-Hz of carrier-to-noise density moving from open ground to under canopy, while median multipath metrics roughly tripled. ESA's Navipedia names the RTK-specific consequence, that the method "needs continuity in the tracked measurements to avoid re-initialization of the phase-ambiguity filters", which obstructions repeatedly break.

This is where a mask earns or loses its keep. It rejects epochs on geometry alone, so it cannot catch a canopy failure where the geometry stayed good. It guards one failure mode; it is not a quality gate.

What corrections can and cannot fix

A correction stream removes the errors common to your rover and the reference station. ESA lists them: satellite clock bias, orbital error, ionospheric and tropospheric delay all cancel in differential processing. It is equally explicit that "the main errors left without correction are multipath, interference and receiver thermal noise".

That split explains the indicators. Corrections shorten convergence and tighten a fixed solution, and their benefit scales with baseline, which is why distance to the reference station drives GNSS accuracy and why our network RTK coverage assigns the nearest base. What they cannot do is clean up a reflected signal, and u-blox concedes the point by making its own accuracy specification conditional on antenna, multipath conditions and satellite visibility.

Age is the failure mode people miss. A u-blox rover stops using corrections older than 60 seconds by default and drops out of RTK, expressly "to prevent the computation of grossly misleading differential solutions". Our guidance is tighter: full performance under one second of latency, more float between one and three, meaningful loss beyond three. For hardware, our guide to choosing a GNSS receiver covers the tradeoffs, and RTK vs PPK covers real-time against post-processed workflows. The same four indicators apply whether you run RTK for surveying, RTK for drone mapping or RTK for robotics.

FAQ