02Statistical evidence

Dataset
SWORT-FL-0142
System
Flight record, 6 channels
Mode
Practice
Session
Local
Dossier

Baseline characterisationDetermine how much variation is normal before classifying anything as deviation.

Illustrative development media The aircraft is modelled. The telemetry is recorded.

Plate

Mission 02 · Foundations

How noisy is normal, and what a threshold cannot tell you.

Three hundred and four seconds of a real flight log. Something happens in it. Before you can say anything about that something, you have to be able to say what this aircraft's instruments do when nothing is happening at all, and that is a number you are going to work out by hand from twelve samples before the machine is allowed to help.

RecordRecorded
FlightSWORT-FL-0142
Samples
Duration
Nominal rate
Channels

Stage

XYour own channel Same procedure, different world
·How you got here Debrief

Not whether you were right. The order you did things in, and what you decided at each point, assembled from what you actually recorded.

    Show the derivation Every stage in full, with the written argument
    BTwelve samples By hand

    Here are twelve consecutive readings of the tracked satellite count, taken from a stretch of the flight where nothing notable is happening. Work out the mean, then the spread. The arithmetic is deliberately small enough to do without a calculator, and doing it once by hand is the difference between sigma being a quantity and sigma being a word.

    Sum of the twelve ·

    Add them, then divide by twelve.

    satellites
    satellites

    Subtract the mean from each value, square each difference, add them up, divide by eleven, take the square root. Dividing by eleven rather than twelve is not a typographical error and section C is where it starts to matter.

    CTwelve to thousands Same calculation

    Those twelve values were not special. They were the first twelve of a great many, and the calculation you just did by hand is the calculation that runs on all of them. Watch what happens to your answer as the sample count grows. Nothing about the method changes.

    Samples·
    Mean·
    Standard deviation·
    Moved from the previous step by ·

    The estimate from twelve samples is not wrong. It is uncertain, and by an amount you can also compute: the standard error of a sigma estimate falls roughly as one over the square root of the sample count, which is why twelve samples move it around by tenths and eight hundred barely move it at all. Every threshold you set from now on inherits the uncertainty of the sigma you set it from.

    DSeries and distribution The same samples, twice

    A time series and a histogram of the same channel are not two pieces of information. They are one set of numbers, sorted two different ways. Drag across either one and watch the other.

    Baseline window. Drag to select, click once to clear.

    Distribution, with the mean and one, two and three sigma marked.

    Three sigma either side of the mean contains most of what this instrument does when it is behaving. That is an observation about this record, not a law: the interval only covers the fraction people quote if the quantity is distributed the way people assume, and a satellite count is an integer that cannot go below zero. Check the histogram before you trust the rule.

    EThreshold Detection, not explanation

    Now set a line. Anything past it gets flagged for a human to look at. Drag the handle in the plot itself, because a threshold that is typed into a box somewhere else is a threshold nobody can see against the data it applies to.

    Threshold ·
    Line sits at·
    Flagged in the baseline window · Where nothing is wrong. These are false alarms.
    Flagged in the whole flight ·
    First flagged at·

    A threshold detects a deviation. It does not explain one. Move the line down and you catch the event sooner and cry wolf in quiet air. Move it up and the quiet air is clean and you find out later. Neither position tells you what happened, and there is no setting that does.

    Switch the channel and watch how differently the same rule behaves. Bus voltage is continuous, so the flagged count moves smoothly as you drag. The satellite count is an integer with four values in it, so the count does not move at all and then jumps by hundreds: there is no threshold between those two states, because there are no measurements between them. Choosing which channel to watch is a bigger decision than choosing where to put the line.

    FOne channel, then more Claim before evidence

    Here is the position channel on its own, which is how most people meet a flight log for the first time. Look at it, and say what you think happened.

    Your initial explanation

    GAnalysis record Artifact

    What you computed, what you set, what you first thought and what you thought afterwards. The last two are recorded separately and both are kept, because a position that changed when evidence arrived is the thing this mission is actually assessing.

    AnalysisNot recorded

    SWORT

    SWORT designs and builds flight computers, GNC hardware and embedded avionics. The Student Research Program is how we bring people into that work.