Security in depth · Unit 28 · Lesson 11 of 11
Case study: decide whether a change is ready
Review a fictional release packet for logging, session handling, dependency changes, and recovery before making an evidence-based decision.
Helpful before thisSecure development: requirements to evidence
After this lesson you can
- Translate a release promise into observable acceptance criteria.
- Separate failed requirements from evidence that does not cover the candidate.
- Justify acceptance, a hold, or a verified reduction in release scope.
The fictional Lantern club is updating its member portal. The change adds support diagnostics, a “sign out this device” feature, a library update, and a database migration. The release owner asks whether it is ready. You receive the packet below; there is no real system to contact and no requirement to execute anything.
An acceptance criterion turns a promise into an observable condition. “Support can diagnose errors safely” is a useful aim, but reviewers need to know which information is permitted, what event should be recorded, and how the candidate behaves. A version number or a green report is useful only when connected to the question being decided.
Artifact A: the release contract
The club approves four requirements for this release. Logs may contain a non-secret operation reference, event type, outcome, and time; they must omit usable session credentials and member document contents. Device revocation must stop that device’s protected actions within sixty seconds while intentionally retained sessions continue. Sixty seconds is this exercise’s agreed limit, not a universal standard.
The dependency review must cover the exact candidate’s component versions and explain any applicable unresolved finding. Recovery must restore a supported application and compatible data without silently losing accepted member changes. The recovery owner must know when to act. These requirements allow meaningful tradeoffs, but changing them needs an explicit decision.
Artifact B: the initial evidence packet
| Area | Supplied evidence | Review status |
|---|---|---|
| Logging | Normal events use the allowed fields; an error handler also stores the usable session value | Failed confidentiality criterion |
| Sessions | Renewal stops immediately; existing access credentials remain accepted for up to twelve minutes | Failed sixty-second criterion |
| Dependency | A clean report covers the old library version; the candidate uses a newer version | Candidate evidence missing |
| Recovery | Last month’s rehearsal predates a column rename required by this candidate | Compatibility not established |
The session result does not mean revocation did nothing. It ended renewal, but failed the promise about already issued credentials. Similarly, the old dependency report remains evidence about its own version. It neither clears the candidate nor proves that the update contains a vulnerability.
For logging, distinguish the stored event from a redacted screenshot. Hiding a value in the review packet would not remove that value from the application’s logging behavior. The reviewer can describe the prohibited field without copying a real secret. For recovery, restoring an older program is insufficient if it cannot read the database produced by the new release.
Make the first decision
Hold this candidate. Two requirements fail and two lack relevant evidence. This is stronger reasoning than “some checks are red,” because it identifies the consequences and the missing coverage. Assign a follow-up to each affected requirement instead of asking for an unspecified amount of additional testing.
An exception record is not itself a compensating control. An owner and review date help accountability, but do not prevent session values from being recorded or shorten accepted access. A narrower candidate could remove the incomplete changes, provided evidence shows the real service behavior and data compatibility match that reduced scope.
PredictThe release owner proposes hiding the new device button while leaving the new session behavior enabled. Does that establish a safe reduction in scope?
No. The interface change does not establish what the service accepts or what existing users experience. A narrowed candidate needs a precise description of its actual behavior and evidence for the requirements that still apply. Delaying the complete change is another defensible option.
Artifact C: a revised packet
The revised candidate removes session values from the error path. Its supplied review covers normal and error logging paths, and synthetic event samples contain only the allowed fields. A device-revocation result shows that each accepting component rejects the revoked session within twenty seconds; intentionally retained sessions still work.
A fresh dependency inventory and review identify the exact candidate and report no applicable unresolved findings within the stated coverage. Recovery now has a documented compatible migration plan and a rehearsal using the candidate’s schema and synthetic member changes. The expected changes survive recovery, and a named owner has the decision criteria and access needed to respond.
These observations close the four stated gaps for this exercise. Acceptance of this defined candidate is justified on the supplied evidence, with the review scope and any other known limitations recorded. Acceptance does not establish that every future dependency is safe or that all failure combinations were tested.
Write a decision another reviewer can use
Name the candidate, requirements, evidence, decision, and accountable owner. Distinguish observed results from expected behavior and identify what would invalidate the decision, such as a later session-policy change or another schema migration. Retain useful non-secret evidence so maintenance can build on it. Recovery helps repair a release; it cannot recall information already disclosed by one.
Terms you met
Check yourself
No timer. No penalties. Read the explanation and try again whenever you like.
This lesson’s questions have changed. Your reading progress is saved; review the updated questions.
-
The initial error handler stores a usable session value. The review screenshot hides it. Which conclusion is justified?
Show the answer
Correct answer: The criterion still fails until the logging behavior omits the prohibited value. The requirement concerns stored events, not only the review packet.
-
Revocation ends renewal but existing credentials remain accepted for twelve minutes. What should the review say?
Show the answer
Correct answer: Renewal ended, but the stated protected-access limit failed. This preserves what worked while identifying the requirement that did not.
-
A clean dependency report covers a version older than the candidate’s. What follows?
Show the answer
Correct answer: Review the exact candidate; the old report establishes neither its safety nor a specific defect. Evidence must match the component versions being accepted.
-
The revised packet closes all four stated gaps. Which acceptance statement is appropriately limited?
Show the answer
Correct answer: Accept this candidate against these criteria, recording scope and conditions that require renewed review. The decision is supported while its practical limits stay visible.
Try it
- WritePrepare a one-page review using only the fictional packet below. Make four rows for logging, sessions, dependencies, and recovery. For each, write the criterion, evidence, verdict, and necessary follow-up. Add a release decision for both the original and updated packets, with one limitation that acceptance does not remove.