Systematic Error Simulator

What Is the Systematic Error Simulator?

How to use the systematic error simulator

1. Read the four cards before you touch anything. They are already answering for a micrometre with a 20.000 um zero error, no gain fault and 4.000 um of scatter, taking nine readings of a 5.000 mm block: Systematic error (bias) 20.000 um, Random error of the mean 1.333 um, Total error (rms) 20.044 um and Repeat ceiling 1 reading.
2. Read the sentence strip under the drawing, because it is the only place the lab gives advice. On that state it says The instrument is the limit. More readings will not help; check the zero and the gain against a standard. It has exactly three forms across every state, and which one you get is decided by the bias share in the grid, 99.56 % here.
3. Compare the two brackets in the top band, not the two numbers. One runs from the solid true 5.0000 mm line to the dashed long-run mean 5.0200 mm line and is labelled the bias, 20.000 um; the other is +/- SE, 1.333 um about this sample’s mean, and it spans two standard errors because it is a plus-or-minus. Both are drawn through the same pixel mapping on purpose, so what you are reading off is a real ratio and not two pictures side by side.
4. Now walk Readings from end to end and watch which cards move. The first card does not move at all. The second falls from 4.000 um to 0.400 um, the third from 20.396 um to 20.004 um, and the bias share climbs from 96.15 % to 99.96 %. A hundred times the readings, and the answer is still twenty micrometres out.
5. Put Readings on its bottom rung and look at the middle band. With one reading there is no trend to plot, so the band says 1 reading: no trend to see rather than drawing an empty frame, and the grid cell for the scatter in this sample reads a guarded 0.000 um because one reading has no degrees of freedom to spare.
6. Switch the instrument off entirely: put Zero error and Gain error on their middle rungs. Both ladders carry an exact nought there, so either fault can be turned off while the other stays on. The first card goes to 0.000 um, the ceiling card to no ceiling, the verdict flips to Your hand is the limit. More readings will help: four times as many halves this. and the upper bar in the error budget stops being drawn.
7. With both faults off, walk Readings again. Now the total error is the standard error alone and it falls from 4.000 um to 0.400 um — exactly tenfold — because there is nothing else left in the budget. This is the only condition in which the usual advice to take more readings is simply correct.
8. Turn the gain fault back on and move True value instead. A gain error is a percentage of what you are pointing at, so moving the first slider now moves the bias. Set 2.00 % and the bias runs from 0.800 um on 40 micrometres of foil to 1.00000 mm on a 50 mm block without the instrument changing at all.
9. Press New sample three or four times. The dots jump, this sample’s mean moves through 5.0186 mm, 5.0203 mm, 5.0230 mm and the scatter in this sample through 4.103 um, 3.523 um, 3.135 um, while all four cards stay byte-identical. Nothing about the instrument depends on which readings you happened to get.
10. Press Reset when you have finished. It puts the five sliders back to 5.0000 mm, 20.000 um, 0.00 %, 4.000 um and 9 readings, returns the sample index to its first pool, and replays the opening settle so the marker eases in rather than cutting to its place.

Dragging is the right way to find out which readings move together and a poor way to land on a particular state, because all five controls step by whole rungs. The five preset buttons above exist for that reason, and the written systematic error guide is where the subject itself is argued from scratch, with the problems worked step by step.

One thing this panel cannot do is take a figure that is not on a rung, and it cannot run the relation backwards at all. The systematic error calculator takes typed numbers and solves in four directions, including the two questions this lab never asks: what bias a measured total error and scatter imply, and how many readings a target total error would need.

Worked example: change one thing at a time

Nothing in the three tables below was worked out here. Each row is a state the lab was put into through its own controls and then read back from, cell by cell, so every string in them is the panel’s own. The first table changes exactly one slider and holds the other four still.

One: one instrument, every rung of the readings ladder

The instrument is the opening one and it never changes: a 20.000 um zero error, no gain fault, 4.000 um of scatter, measuring 5.000 mm. Only the fifth slider moves, from 1 reading to 100 readings.

The second column is the same string eight times, and that is the whole table. A hundred times the work moved the random error of the mean by a factor of ten, exactly as four times as many readings halving it requires, and moved the total error from 20.396 um to 20.004 um. The repeat ceiling said 1 reading before any of that was attempted.

The ratios behind those two numbers are asserted rather than estimated, and they are smaller than they look. Going from nine readings to a hundred — eleven times the work — removes 0.2015 per cent of the error. All hundred together, against a single reading, remove 1.9223 per cent. Under a quarter of one per cent is the honest way to say the first of those.

The last column is the one readers mistake for a fault. The scatter in this sample wobbles down the table — 3.883 um, 4.909 um, 4.103 um — because it is a sample statistic on n minus one degrees of freedom and not the setting. The setting is 4.000 um throughout. How many digits of any of these figures a written answer may keep is a separate question, and the significant figures guide is where it belongs.

Two: the five published states

These are the presets with the readings their buttons produce. Read the last two columns across: the bias share and the verdict are the two that decide whether more readings are worth taking, and they disagree completely from row to row.

Rows two and three are the same instrument, and they are the strongest argument on this page. Same 2.00 % gain fault, same 4.000 um scatter, and the bias goes from 0.800 um to 1.00000 mm — a factor of 1,250 — purely because the thing being measured went from 40 micrometres of foil to a 50 mm block. One of them needs 25 readings before the instrument becomes the limit; the other is beyond help at the first.

Row two is the only one of the five where the verdict strip takes its middle form. The bias is 0.800 um, the random error of the mean is 0.800 um, the share is exactly 50.00 %, and the ceiling is exactly 25 readings — so the strip under the bars reads at the ceiling rather than before or past it. It is the one state of the five where both terms genuinely matter.

Row four is the control case and row five is the twist. Row four has both faults off, so the total is the standard error alone and the ceiling card reads no ceiling. Row five has two faults that happen to cancel at the length on screen, which produces the same 0.000 um from a completely different situation — and the third table is what that one is really hiding.

Three: two faults, and the one length where they cancel

This is the fifth preset with only the first slider moved. The instrument is fixed throughout: a -5.000 um zero error, a 0.50 % gain error, 2.000 um of scatter, nine readings. Nothing about it is touched; only the size of the thing being measured changes.

Calibrate at 1.0000 mm and this instrument certifies itself as perfect. That row reads 0.000 um of bias and a long-run mean of 1.0000 mm against a truth of 1.0000 mm: the check finds nothing at all. Take the same instrument to the 25 mm standard and it reads 25.1200 mm. A one-point calibration at the wrong length is worse than no calibration, because it comes with a certificate.

The sign reverses across that length, and the two error columns disagree about where the trouble is. The absolute bias is worst at the top of the ladder, at 245.000 um; the relative bias is worst at the bottom, at -12.00 % against 0.49 %, because the fixed offset dominates where the measurement is small. The same instrument reads low on a foil and high on a block.

This is the common case rather than a curiosity. Of the 64 pairs of zero error and gain error the two middle sliders can set, 22 have a bias that changes sign somewhere across the true-value ladder, which is 34.375 per cent of them. All three of those counts come from the reference script sweeping the grid, not from a sample of it.

Formula and symbol reference

The band at the head of this page carries the total-error relation with the definition of the bias beside it, because the first on its own hides the fact that b is built from two independent faults with completely different behaviour. Every reading on the panel is one of those two expressions, a count derived from their ratio, or a share.

The symbol column below uses the panel’s own names, spelled the way the lab’s own cards spell them in plain ASCII, with sigma and sqrt written out rather than drawn. The lab switches from micrometres to millimetres at exactly one millimetre, which is why one reading can read 245.000 um and the next 1.00000 mm; the SI units guide is where those prefixes come from, and the unit conversions guide is the one to keep open if you are working in anything else.

If you read one row twice, make it the bias share. Everything else on the panel is a quantity; that one is a decision, and the verdict strip is simply it put into words. Above about nine tenths the instrument is what is left and no reading count helps; below about a tenth your hand is, and four times as many readings really do halve what you have.

The physics: why averaging cannot touch a bias

Leave the panel on its opening state and ask what the dashed line in the top band is. It is the long-run mean, which is where the average of your readings is going: not the truth, but the truth plus the bias. The dots scatter about that dashed line and never about the solid one, and no number of dots changes which line they are scattered about.

That is the whole of it, and the middle band is the proof drawn rather than asserted. The envelope there is two sigma over the square root of the reading count, and it closes as the count rises — onto the dashed line. The running mean walks inside it, converging neatly and confidently on the wrong answer, which is why the band labels the envelope the envelope narrows onto the long-run mean instead of leaving a reader to assume otherwise.

The two terms therefore cannot be added head to head. They are independent, so they combine as the legs of a right-angled triangle, and shrinking the shorter leg barely moves the hypotenuse. That is the arithmetic behind every row of the first table: at nine readings the bias is fifteen times the standard error, and squaring both before adding is what turns a fifteen-to-one ratio into a 99.56 % share.

The repeat ceiling is an inverse square in the bias, which is the least intuitive thing on the panel. Halving the bias quadruples the count at which the scatter has fallen to meet it, so a better instrument stops rewarding repetition sooner, not later. Hold the scatter at 4.000 um and the foil preset’s 0.800 um bias gives a ceiling of 25 readings; a 1.000 um bias gives 16, doubling that to 2.000 um gives 4, and doubling again to 4.000 um gives 1.

Seeing the fault costs about four times what tolerating it costs, while the scatter is the larger of the two. The grid row for the readings needed at two sigma rounds up four times the squared ratio of the scatter to the bias; the ceiling card rounds up the same ratio squared, without the four. On the foil preset that is 100 readings against a ceiling of 25 readings — but each is rounded up on its own, so the low corner of the grid reads 38 readings against 10 readings, and the opening state reads 1 reading on both.

That asymmetry is why a bias usually goes unnoticed rather than unmeasured: the count at which it starts to dominate your answer arrives long before the count at which it becomes visible in your data. Between those two counts you are being misled by an instrument whose fault your own readings cannot yet show you.

There is exactly one escape, and it is not on this panel. A fixed offset can be divided by the number of events you measure across, which is a change of measurement design rather than of statistics: time twenty swings of a pendulum once and a stopwatch’s fixed offset is divided by twenty, where timing one swing twenty times leaves it whole. The pendulum period guide is where that case is worked, and the pendulum period calculator gives the period it is used to measure. Nothing on these five sliders can do it.

Where this lab breaks down

Every one of the 32,768 states the five sliders reach is answered, and the arithmetic is not where this panel can let you down. What it cannot do is model a real measurement, and what it refuses to do is treat its own idealisations as facts. The first item below matters more than the rest put together, and it is the one no competing page seems to carry.

The random and systematic split is the right thing to learn and is not how metrology books the arithmetic

Read this one before any of the others. The NIST summary of Technical Note 1297 classifies the components of an uncertainty by the method used to evaluate them — Type A from the statistical analysis of a series of observations, Type B by any other means — and not by whether the cause wobbles. Put the zero error on this panel into a real uncertainty budget and which category it lands in depends on how you came by the number, not on anything the instrument did. This lab draws the random and systematic picture because it is how the subject is taught and how it is understood, so do not be surprised if a calibration certificate is not written in those terms.

Subtracting a bias does not delete it

The same summary says a measurement equation should express a measurement process rather than simply a physical law, and that the quantities in it include the corrections. Correcting a reading therefore replaces a bias with the uncertainty of the correction, which is smaller but is not nothing. This panel takes the bias as an exact given and prints what it costs; it has no control for how well the correction itself is known, and no reading here should be read as saying a corrected measurement is exact.

The bias here is constant, and real faults drift

The lab’s bias is the same in every reading by construction, which is what makes it invisible to repetition. A drifting fault is not that: it both shifts the mean and inflates the apparent scatter, which is exactly why it is the easier fault to catch — the readings trend with time rather than varying randomly about a mean. A constant offset adds precisely nothing to the scatter in this sample, which is the asymmetry behind the entire subject and the reason there is no drift slider here.

Repeats with the same instrument and the same observer cannot see the fault they share

Nothing in the panel knows the true value except the panel. On a bench the only two routes to a fixed systematic error are measuring a quantity whose value you already know, or comparing your readings with readings made on different apparatus known to be more accurate. The first slider here is the luxury a simulation has and a bench does not, and the third table is what happens when the known quantity you chose was the one length at which your two faults cancel.

The standard error assumes the readings are independent, and nothing warns you when they are not

Sigma over the square root of n is a statement about independent readings. Nine measurements by the same hand in the same five minutes are correlated, and the second card is then optimistic by an amount the formula gives no hint of. There is no correlation control on this panel, so every standard error it prints is the best case. That makes the bias share printed beside it a lower bound as well, which cuts the same way: the instrument’s share of a real budget is usually larger than the figure here, not smaller.

Sigma is set here and estimated in practice

The fourth slider sets the scatter of a single reading as a known quantity. On a bench it is estimated from the sample itself, on n minus one degrees of freedom, and it is uncertain by roughly one over the square root of twice that — about a quarter of its own value at nine readings. The last column of the first table is that uncertainty on display: the setting is 4.000 um throughout and the sample statistic wanders between 3.649 um and 4.909 um without the instrument changing at all.

The scatter is normal here, and a dial rounds instead

Readings are drawn from a normal distribution. A quantised instrument does something different: rounding to the nearest step is a uniform error whose standard deviation is the step over the square root of twelve, which for a micrometre reading to a thousandth of a millimetre is about 0.2887 micrometres. That is 57.7350 per cent of the smallest scatter rung on the fourth slider and below it everywhere, so the normal model stays defensible across this panel’s whole range — but below one step of resolution a real scatter can read as exactly zero while a rounding bias is still entirely real.

A gain error is a percentage of a quantity, and a scale has its own zero

The third slider multiplies the true value, which is the right model for a length measured from a hard zero. It is not the right model for a reading on a scale whose zero is a convention rather than an absence of the quantity: two per cent of a temperature in degrees Celsius and two per cent of the same temperature in kelvin are different faults, and only one of them is what a sensor’s gain actually does. The absolute zero guide is where that distinction lives. Say which zero you mean before you quote a percentage.

None of this is the limit the uncertainty principle sets

Everything on this panel is a property of an instrument and could in principle be calibrated away. The uncertainty principle guide is about a limit nature imposes on what is simultaneously knowable, and the uncertainty principle simulator draws it in two conjugate axes. A figure count is a third thing again — a statement about how a number was written. All three limit what a measurement can tell you and no care about one of them buys anything against another.

The sliders are ladders, and the lab runs one way only

All five controls step by whole rungs, so the panel reaches 32,768 states and nothing between them. It also runs forwards only: you set a fault and read what it costs. There is no box to type a measured total error into and no way to ask for a bias back. Every figure on this page is arithmetic on five slider positions and one seeded sample, nothing here was measured from a real instrument, and the lab prints its own assumptions in a fixed line under the controls rather than leaving them implied.

Where the systematic error simulator is actually used

Deciding how many readings to take before you take them

This is the question the panel exists for, and the answer is on one card. Put your instrument’s known bias and its single-reading scatter on the second and fourth sliders and read the Repeat ceiling. At 20.000 um against 4.000 um it is 1 reading; cut the bias to a twenty-fifth and it becomes 25 readings. Everything past that count is work you are doing for the look of the thing.

Working out whether a fault is even detectable with the data you have

The grid row for the readings needed at two sigma is the one to quote when somebody asks why a known fault never showed up in their results. It is about four times the ceiling wherever the scatter is the larger of the two quantities, each count being rounded up on its own: 100 readings where the ceiling is 25 readings, and 38 readings where the ceiling is 10 readings at the low corner of the grid. If your run was shorter than that figure, your data was never going to see the fault, and the absence of evidence in it is not evidence of absence.

Pricing a calibration against more bench time

Load the first preset and the trade is on screen in two cards. A hundred readings have taken the random error of the mean to 0.400 um while the total error sits at 20.004 um, so the hundredth reading is defending four tenths of a micrometre against twenty. Checking the zero against a standard is the only thing in that picture that moves the big number, which is what the verdict strip says in words.

Catching a calibration that was done at the wrong size

The third table is this case and it is the sharpest thing on the page. One instrument, two faults of opposite sign, and a single length at which they cancel exactly: check it there and it reads 1.0000 mm against a 1.0000 mm standard and passes. The same instrument is 245.000 um out at 50 mm and -4.800 um out on 40 micrometres of foil. Two standards at different sizes is the cheapest defence, and 22 of the 64 fault pairs here need it.

Explaining why a careful experiment still came out wrong

Set both faults off and the panel behaves exactly as a first course promises: the error falls tenfold from one reading to a hundred and the mean converges on the truth. Turn one fault on and it converges just as neatly on something else. The true value has to be counted in with the three fault sliders here, because a gain fault’s bias is a percentage of whatever you point it at: of the 4,096 states those four sliders reach, 152 have no bias at all, and 2,808 of the remaining 3,944 are already bias-dominated at a single reading, about seven in ten.

Reading a quoted constant the way it was meant to be read

Published constants come with an explicit uncertainty rather than a figure count, which is exactly the distinction this panel draws between a setting and a reading. The physics constants guide keeps the values with their uncertainties attached. Nothing on this page produces such a figure for a real instrument; it shows you what shape the budget behind one has.

Telling an instrument problem apart from a notation problem

If more readings do not help, the trouble is in the hardware and this panel is the picture of it. If the trouble is in how many digits a written answer is entitled to, it is a different subject with its own tools: the significant figures calculator gives the half-unit a written number claims and the significant figures simulator shows which digits of a result a figure count supports. A figure count is a coarse stand-in for precision and never the precision itself.

Where to go next

A panel can put you in a state and let you move one thing at a time, and that is the end of what a panel is for. The systematic error guide is where the subject is written out: what the two kinds of error are, why repeating a measurement cannot touch one of them, where the formal classification parts company with the teaching one, and the problems worked step by step.

Two posts own the distinctions this lab is careful not to blur. The significant figures guide is about how a number was written, and the uncertainty principle guide is about a limit nature sets rather than one an instrument does. A systematic error is a third thing again, and it is the only one of the three a calibration can do anything about.

Three more pick up what this lab uses but does not explain. The SI units guide is where the millimetre and the micrometre come from, the unit conversions guide is the one to keep open if your instrument reads in anything else, and the physics constants guide is where quoted values keep their uncertainties attached.

Two sit one step further out. The pendulum period guide carries the one classic case where a fixed offset really can be beaten, by timing twenty swings instead of averaging twenty timings, and the absolute zero guide is where the question of which zero a percentage is measured from is settled.

On the tool side, the systematic error calculator takes typed figures and solves in four directions, the significant figures calculator works on the written number, the uncertainty principle calculator gives the quantum floor, and the pendulum period calculator gives the period behind the stopwatch case.

Three neighbouring pictures sit beside this one: the significant figures simulator, the uncertainty principle simulator and the unit conversions simulator. Beyond them there is the simulation library, an index of calculators, and the blog for everything written out at length.