All lessons Leer en español

Security in depth · Unit 29 · Lesson 9 of 9

Case: the booking service is down

Separate service impact from compromise claims, then decide what counts as a verified recovery.

10 minready

Helpful before thisTabletop: practice a security incident togetherRecovery: how much time and data can you lose?Digital evidence: the story and its limits

After this lesson you can

  • distinguish observed unavailability from evidence about its cause
  • sequence recovery around dependencies while preserving relevant evidence
  • report recovery measurements and remaining uncertainty without overstating success

The situation at the fictional Alder club

Alder runs a small equipment-booking service. At 09:00, its agreed start of disruption, staff cannot complete a new booking. All times below are invented UTC times; assume the stated clocks are synchronized. You are preparing the incident lead’s decision record using supplied evidence only. Availability describes whether people can use the service. Compromise is a separate claim about unauthorized activity; an outage alone does not establish it.

The owner has agreed an RPO of thirty minutes and an RTO of sixty minutes. The recovery point objective (RPO) limits tolerable loss of recent data, expressed as time. The recovery time objective (RTO) limits the time allowed to restore the defined service. Recovery ends when staff can sign in, create and retrieve bookings, pass the agreed access checks, and account for queued requests with unresolved items explicitly tracked. A working public status page does not satisfy that endpoint.

Stage 1: read the evidence without completing the story

Received Fictional evidence Limit
09:03 Support confirms failed booking requests and staff sign-in Confirms impact, not cause
09:06 Administrative collector’s last heartbeat was 08:40 Recent security visibility is incomplete
09:09 Identity provider reports an issue beginning at 08:58 Provider statement, not proof of every local failure’s cause
09:12 Local report records an administrative change at 08:55; ticket shows planned work at 09:30 Approval and timing context need review
PredictThe dashboard shows no alerts. Which statement belongs in the first update: “No compromise,” “Confirmed compromise,” or “Cause unconfirmed, visibility incomplete”?

The third. The team has evidence of service impact and missing telemetry. A provider problem and an unauthorized change are possible hypotheses, not established conclusions. Neither silence nor the administrative report resolves them.

The first update should name the affected booking function, the incident coordinator, and the next update time. Avoid converting a provider’s statement into your own confirmed finding. Preserve relevant reports and their context through the organization’s authorized process. Record what a decision depends on: an investigation into approval can continue while the recovery lead assesses usable options.

Stage 2: follow the dependencies

At 09:15, the recovery lead reports that the current booking store fails application consistency checks. The reason remains under investigation. Under the agreed plan, a verified recovery point contains committed data through 08:40; its keys and recovery access are available independently of the failed identity provider. The incident lead approves recovery after the evidence owner records the preservation decision and its limits. These are supplied exercise facts, not capabilities established by this lesson.

Recovery with evidenceIdentity access and booking data are two parallel dependencies that converge on service acceptance checks. Preserving and assessing evidence is parallel work, not a stage that begins after recovery finishes.Recovery dependenciesIdentityaccessBookingdataVerify the serviceBoth are requiredIn parallel: evidencePreserve and assess
Identity access and usable booking data are parallel recovery dependencies. The agreed service checks need both; evidence work continues alongside recovery.

At 09:20, an approval ledger and the responsible supervisor confirm an emergency change window of 08:50-09:00 for the reported administrative change. This reduces concern about that particular timing discrepancy. It does not establish the cause of the consistency failure, validate every administrative action, or fill the telemetry gap.

The booking database passes its component checks at 09:22. Staff sign-in remains unavailable until 09:34. Restoring one dependency is progress; presenting that milestone as completed recovery would hide the remaining blocked workflow. NIST’s contingency-planning guidance helps frame this dependency problem, while its incident-response guidance keeps analysis and recovery within the same coordinated process.

PredictCould the incident lead report recovery at 09:34 because staff can now sign in?

Not yet under the stated agreement. Staff must also verify the booking workflow, access restrictions, and treatment of queued requests. Authentication success does not prove that the application uses the restored data correctly.

Stage 3: decide what success proves

At 09:42, the agreed checks pass. Test bookings can be created and retrieved, the checked staff roles receive the expected access, and queued requests are accounted for in a reconciliation register. That register explicitly tracks unresolved bookings from the data gap; passing the acceptance checks does not assert that every historical record was recovered.

The measured recovery duration is 09:42 minus 09:00: forty-two minutes, within the sixty-minute RTO. The recovery point is twenty minutes before disruption, within the thirty-minute RPO. Twenty minutes is a time window, not a count of lost bookings. Establishing actual losses requires reconciliation against appropriate independent records and the agreed business process.

Model handoff

“Booking service accepted at 09:42 after coordinated identity and data recovery. Recovery duration forty-two minutes against the sixty-minute RTO; recovery-point gap twenty minutes. The reported administrative change has corroborated emergency approval. Cause of the consistency failure, the collection gap, and unresolved historical bookings remain open, with named owners and the next review time.”

An owner could instead authorize limited service earlier if its functions and limitations were explicit; that would be reported as a separate milestone, not silently substituted for this endpoint. A suspected unsafe recovery point would require another decision, possibly with a larger data gap. Successful restoration narrows the availability problem. It does not automatically close an investigation or prove the absence of compromise.

Check yourself

No timer. No penalties. Read the explanation and try again whenever you like.

  1. The booking service fails, while its administrative log collector has stopped reporting. What does the quiet security dashboard establish?

    Show the answer

    Correct answer: It cannot establish absence of compromise during the collection gap. Missing telemetry limits the conclusion even though service impact is already observable.

  2. A reported administrative change appears outside a planned maintenance window. What is the best initial response?

    Show the answer

    Correct answer: Preserve the relevant report, seek approval and timing context, and evaluate which recovery decisions depend on the answer. The observation warrants investigation without prematurely deciding cause or intent.

  3. The booking database passes its checks at 09:22, but staff cannot sign in until 09:34 and full acceptance checks pass at 09:42. Which time ends the agreed recovery interval?

    Show the answer

    Correct answer: 09:42. The stated endpoint includes the verified booking workflow and treatment of queued requests.

  4. Disruption began at 09:00, the verified recovery point is 08:40, and acceptance completed at 09:42. What should the handoff report?

    Show the answer

    Correct answer: A twenty-minute recovery-point gap and forty-two-minute recovery duration, with actual missing records and cause assessed separately. Both intervals meet the stated thirty- and sixty-minute objectives, but do not prove zero data loss or establish cause.

Try it

  • WriteWrite a one-page fictional incident handoff with four parts: known facts and source limits; two competing cause hypotheses and useful next evidence; a dependency-aware recovery acceptance plan; and calculated RPO/RTO results with remaining data and investigation gaps. Use only the supplied evidence.
References