Key takeaways
- Write the pilot decision as one question with a named owner and a due date.
- Use two or three real backlog tickets, including one defect and one shared-contract change.
- Grant named access only when a ticket requires it, and keep your normal review path.
- Judge the pilot on finished work, review cycles, rework, and blocked hours.
- Record a pass, conditional pass, or stop decision with conditions and a review date.
Define the pilot decision before the first ticket
A pilot earns its place only when it answers a decision your team has already named. Write that decision as a question, such as whether this developer can own a bounded service change with ordinary review support. Name the manager who will answer the question and the date the answer is due. Without that sentence, two weeks of work becomes a vague impression instead of a decision record.
Keep the pilot question narrow enough that one person can answer it from evidence. A question about overall culture fit invites a personality judgment, while a question about review response invites a checked record. State what would count as a pass, a conditional pass, and a stop. Share those definitions with the developer before the first ticket moves.
Choose work that resembles the real backlog
Select two or three tickets from the actual backlog and remove anything that cannot be shared safely. The work should include one ordinary feature, one defect with a reproducible path, and one change that touches a shared contract. Redact customer data and replace production credentials with sandbox access. A pilot built from invented exercises tells you how someone performs a test, not how they handle your system.
Write the acceptance criteria before the developer starts, including the check that will prove the change works. Link each ticket to the file area, the reviewer, and the environment where it can run. If the ticket depends on a decision nobody has made, treat that as a finding about your process. A blocked pilot often reflects an unready backlog rather than a weak developer.
Set the access and review boundary
Grant named access to the repository, the ticket board, and one safe environment, and record the approval for each permission. Start with read access and add write permission only when the first ticket requires it. Do not share a personal account or a reusable secret to save setup time. The pilot should prove that your normal access path works for an external contributor.
Keep the same review rules you use for employees, including a small pull request and a named approver. The developer should open a draft early when an assumption needs a decision, and the reviewer should answer within the agreed window. Record review comments, requested changes, and the final merge decision in the pull request. A pilot that bypasses review cannot show whether the developer fits your review process.
Measure evidence, not activity
Track finished tickets, review cycles, rework, and blocked hours rather than commits, messages, or online time. A developer who produces one careful change with clear evidence may be more useful than one who opens six thin pull requests. Count how often a ticket returns for missing information or an avoidable defect. Those numbers describe the working system, not a person's worth.
Ask the reviewer to write a short evidence note for each completed ticket. The note should name the behavior changed, the check that was run, the remaining risk, and the next owner if work continues. Keep the notes beside the ticket so the exit conversation can cite them. A summary written from memory later will smooth over the failures the pilot was meant to expose.
Prepare the developer before day one
Send the pilot question, the ticket list, and the access path before the start date. Ask the developer to confirm what they can open and what they still need, then fix gaps before the first working day. A short setup call is useful when it produces a written checklist rather than a conversation. Preparation reduces the chance that two weeks begin with a week of account requests.
Share the repository conventions, the test command, and the review template in the same message. Point to one merged pull request that shows what a good change looks like in this codebase. Explain how questions should be asked and where the answer will be recorded. A developer should not have to infer the team's working rules from old comments.
Keep a weekly checkpoint
Hold two short checkpoints, one at the end of each week, with the manager and the reviewer. Ask what is finished, what is blocked, and what decision is due before the next ticket can move. Compare the answers with the evidence notes instead of accepting a general update. The checkpoint is a chance to correct the pilot while there is still time to use the result.
Record one change after each checkpoint, such as a clearer acceptance check or a faster review response. If the same blocker appears twice, treat it as a process problem and give it an owner. Do not wait until the final meeting to discover that the pilot never had a fair chance. A short correction early is more valuable than a detailed explanation later.
Write the exit decision and next review
At the end of the pilot, hold one meeting with the manager, the reviewer, and the developer. Read the evidence notes aloud, then choose one of the three outcomes defined at the start. Record the reason, the conditions attached to a conditional pass, and the next review date. Do not let the meeting end with an informal promise to continue.
If the pilot passes, add one responsibility rather than opening every system at once. If it fails, preserve the access log, the completed work, and the notes so a later decision has context. Either way, remove permissions that the next phase does not need. A pilot is useful when it leaves a cleaner decision and a smaller permission footprint than it found.
Questions about assessing Philippine developers
How much work should fit in a two-week pilot?
Two or three bounded tickets are usually enough when one is a defect and one touches a shared contract. The aim is a reviewable sample, not a demonstration of speed. Leave room for setup, questions, and one rework cycle.
Should the pilot use production access?
No. Use a sandbox or staging environment with synthetic data and named accounts. Production access should follow your normal approval path only after the pilot decision and a written scope.
What if the developer finishes early?
Ask for a short handoff note and a second review pass rather than adding unrelated tickets. Extra work can widen the sample but also changes the decision criteria midway. Keep the comparison fair by holding the scope.
