Every check explained

Each check is listed under its name as the console shows it. On arrival checks run as each interview lands (and on demand); on demand checks run over a whole form when you choose them in Run HFC checks. Every threshold below is a starting point you can change in the form's quality-check settings; "none" means the limit is off until you set one.

Several checks recognise "don't know" and "refused" answers by their codes. By default these are -98, dont_know and dk for don't know, and -99, refused and ref for refused (the checks that compare enumerators also count 98 and 99). Set them to the codes your instrument uses.

Checks on each interview

Interview duration

An interview much faster or slower than the form's own median, from its start and end times. It flags anything under half or over twice the median (once at least 30 interviews give a median to compare with), and anything outside an absolute minimum or maximum in minutes that you set. Needs: the start and end times mapped.

GPS validation

A reading that is missing, less accurate than you accept, further than a set distance from the survey area's centre, or closer than 10 metres to another interview's point (two supposedly different households in one place). Missing GPS is flagged by default; on a form that collects no GPS that would raise one finding per interview, which is why such a flood is folded into one finding. Settings: the area's centre and maximum distance, the duplicate radius (10 m), the worst accuracy accepted (none).

Straightlining

The same answer given down a battery of questions asked on one scale: every item of a satisfaction scale answered "agree". You define each battery (its questions, and the share of identical answers that counts, by default all of them) and how many batteries must straightline before an interview is flagged (one).

Don't know / refused rate

An interview where too much of the questionnaire went unanswered: more than 15% of the questions counted were "don't know" or "refused". Leave the list of questions empty to count every answered question.

Skip-pattern violations

A question answered that should have been skipped, or left blank when it should have been asked. Each rule is a pair: when a question has a given answer, then another must be answered (or must be blank). It is useful wherever the form's own relevance does not enforce a skip.

Impossible travel speed

Two consecutive interviews by the same enumerator whose GPS points are too far apart for the time between them: faster than 120 km/h by default. Gaps shorter than 10 seconds are ignored.

Missing required answers

An interview arriving without an answer the analysis needs (an ID, consent, the stratum), whatever the form itself requires. You list the questions. A blank answer counts as missing, and you can add codes that should count as empty too.

Interviews outside fieldwork hours

An interview starting at an hour nobody is knocking on doors: before 6:00 or after 20:00 by default, in the workspace's time zone. Set the hours to the project: a market survey may start before dawn.

Late upload

An interview uploaded long after it finished: more than 48 hours by default. Collection is offline by design, so sending in the evening is normal; a week is not.

Contradictory answers

Answers that are each valid alone and cannot both be true: a twelve-year-old with three children, a child born before their mother. Each rule is a set of conditions that must all hold for the interview to be flagged; conditions compare a question with a value or with another question, dates included.

Value outside expected range

A numeric answer outside the range you expect (soft bounds, a warning) or outside what is possible at all (hard bounds, a critical finding). One rule per question, for example a household size expected between 1 and 15 and impossible above 40. Don't-know and refused codes (such as -98, -99, -999) are not treated as values.

Repeat groups filled in the wrong number of times

A roster with too few or too many entries, or not as many as the interview said there would be: a household of five with three members listed. Each rule names the repeat group and a minimum, a maximum, or the question that gives the expected count.

Checks over the whole form

Enumerator productivity

More interviews by one enumerator on one day than is plausible. Days are counted by when each interview happened (its start time, else its end time; the finding says when it had to fall back to the upload time), in the workspace's time zone, with one finding per enumerator-day. Settings: the maximum per day (none until you set it).

Daily output and trend

Enumerator-days that finished under a target number of interviews (none until you set one), and an enumerator whose recent output (the last 7 days) has fallen below 60% of their own earlier average. The last day in the data is not judged, since it is probably still in progress.

Enumerator duration vs. peers

An enumerator whose average interview is far from the rest of the team's: more than two standard deviations, the ipacheck and DIME starting point. It needs at least 10 interviews per enumerator and at least 5 enumerators to compare.

Don't know / refused vs. peers

An enumerator recording far more (or far fewer) non-answers than the rest of the team: two standard deviations by default, with the same minimums. You can also set a fixed limit per enumerator, whatever the team does.

Flag rate vs. peers

An enumerator whose interviews keep coming back marked by the other checks: their findings per interview, two standard deviations above the team's. This is the enumerator scorecard, as a check.

Screening-out rate vs. peers

An enumerator who finds far more households ineligible than their colleagues do, which can be a way of avoiding long interviews. You name the question whose answer ends an interview early and the answers that mean screened out. Two standard deviations, at least 15 interviews each and 5 enumerators by default.

Numeric outliers

Answers far outside the distribution the survey produced for that question: more than three standard deviations from its mean, once 30 answers are in. Choose quantities such as incomes, areas, counts and amounts, not codes or scales. Missing-value codes are left out.

Digit preference (heaping)

Numbers clustering on 0 and 5: estimated rather than measured. A question is flagged when 40% of its answers end in 0 or 5 (an unrounded quantity gives about 20%), and an enumerator when they round two standard deviations more than the team. Values under 10 are ignored, since a household of 5 is not a rounded 5. Choose measured or recalled quantities: ages, weights, heights, amounts.

Question non-response

Questions most of the sample would not or could not answer: more than 25% "don't know" or "refused", or more than 40% left blank, once 30 interviews are in. It points at a question that needs rewording or retraining, not at one enumerator.

Duplicate household ids

The same identifier submitted more than once: a household ID, a respondent ID, a barcode. Letter case is ignored, and placeholder values (na, none, 0, -98, -99) are not compared. In a panel with a case list, a household is counted again only within one round.

Same respondent, different ids

One person recorded against several households: the same phone number, name or national ID under different household IDs. For numbers, the last nine digits are compared, so prefixes written inconsistently still match; values shorter than six characters are ignored.

Fuzzy duplicate answers

Open-ended answers copied between interviews: two free-text answers more than 85% alike. Every answer is compared with every other once, and the finding names the earlier interview it matches. Choose free-text questions long enough to be distinctive.

Benford's law

Whether the leading digits of naturally occurring amounts (expenditure, land area, values) follow Benford's distribution, as real amounts spanning several orders of magnitude do and invented ones often do not. It gives one finding about the form, tested once at least 20 numbers are in, at a significance of 1%. You must choose the questions: ages, codes and scales do not follow the law, and testing them would be noise presented as evidence.

Superseded form version

Interviews still arriving on a version of the form that has been replaced: a phone that never updated. You choose the question that carries the version (formdef_version on most forms).

Sample coverage

Sampled units with no interview, and interviews that are not on the sample list. Joripo never knows the survey's universe, so you upload the sample list (.csv, .xlsx or Stata .dta) and say which column holds the unit's ID and which question holds the same ID. Optionally, coverage is reported by a group (cluster, union, ward): a group is flagged when its coverage is under 80% and it is 20 points behind the rest of the survey, since early in fieldwork everything is uncovered. Groups under 10 units are ignored.

Back-check comparison

Answers from a back-check visit compared with the original interview. The back-check form is another form of the workspace; each back-check is matched to its original interview by an identifier both carry, and the questions you choose are compared exactly, as categories, within a numeric tolerance or within a number of days. It flags an interview whose back-check disagrees on more than 20% of the questions compared, and a question, an enumerator or a back-checker whose disagreement rate is over 10%, once at least 5 households have been back-checked.