A staged plan with a policy before the pilot and measurement beyond self-report.
Cohort sizes and durations below are illustrative.
phase weeks who what has to be true to proceed
--------- ------ ----------------------- -------------------------------
policy 0-2 legal, security, eng approved tool, data classes,
retention terms reviewed
pilot 3-10 2 cohorts of 20, baseline measures taken BEFORE
mixed seniority, access; weekly feedback; no
plus 20 not yet enabled critical security findings
training 9-12 all leads review practice for generated
code; known failure patterns
rollout 13-20 remaining teams in adoption tracked per team;
waves of about 40 incidents and defects watched
measures, compared with the baseline and the not-yet-enabled group:
lead time for changes, review time, change failure rate,
escaped defects, and survey data as ONE input, not the verdict
The baseline is the step most rollouts skip. Without measures taken before access,
any later improvement is unprovable, and the only evidence left is engineers
saying they feel faster, which is real but not what a board will accept as proof.
The group not yet enabled is a comparison, not a control in the experimental
sense: teams differ, and people talk. It still beats a before-and-after alone,
which cannot separate the tool from everything else that changed that quarter.
Say that limitation out loud.
Guardrails sit in the policy phase: which tool, which data may be sent, how long
the vendor retains it, and how generated code is reviewed. The review question
matters more as adoption grows, because faster generation moves the bottleneck to
review, and a team that reviews generated code less carefully than human code
will see it in defect and incident rates some months later.
Change failure rate is in the measures list deliberately. Speed with rising
failures is not a gain, and watching both is what stops a rollout being declared a
success on the half of the evidence that flatters it.