QCHALLenge Program: Women in Tech

• QCHALLenge Program is a scalable, randomized intervention that uses cohort-based portfolio challenges to help Polish women transition into technology-sector roles.
• It employs rigorous evaluation, including randomized admissions, stratification, and peer-review feedback, to compare its impact with traditional mentoring.
• Optimized resource allocation through prioritization algorithms maximizes employment gains at a low cost of approximately $15 per participant.

The QCHALLenge Program is a scalable, randomized-controlled intervention designed to facilitate labor market transitions for women into technology-sector jobs in Poland, developed by Dare IT in collaboration with Stanford researchers. With a per-participant cost of approximately $15 and a curriculum built around cohort-based portfolio challenges, the program aims to help women with existing technical skills but lacking paid tech experience build résumé-ready demonstrable competencies. QCHALLenge employs randomized admissions and robust evaluation methodologies to quantify its effectiveness relative to traditional one-on-one Mentoring, and explores prioritization algorithms to optimize aggregate impact under strict capacity constraints (Athey et al., 2022).

1. Program Structure and Implementation

QCHALLenge operates as a fully online, cohort-based alternative to individualized mentoring. The essential design elements are:

• Target Population: Women not currently in paid technology roles, with self-reported English B1 proficiency and tool fluency (Figma for UX or React for front-end).
• Admissions and Capacity: Of 500 eligible applicants in December 2021, the first 100 were admitted ex ante, and the remaining 200 slots filled by randomized lottery among the next 400. Cohort capacity was strictly 300 participants, with oversubscription handled via randomized admission.
• Curriculum: 16 weeks, comprising a two-week kickoff, twelve weeks of six biweekly “challenges” (mini-projects drawn from live business cases), and a two-week wrap-up.
  • UX track examples: persona definition, user journeys, wireframes, prototypes, presentations.
  • React track examples: UI atomic components, REST API modules, TypeScript integrations, form and chart development.
  • Each challenge task was designed to require approximately eight hours over two weeks.
• Peer Review and Feedback: Regular live review sessions led by volunteer experts replaced one-on-one mentoring. Slack workspace enabled group communication; small peer groups were formed randomly for collaboration.
• Delivery Cost: Total incremental outlays for 300 participants approached $4,200, yielding a unit cost of ≈$14–$15.

2. Experimental Evaluation and Methodology

Randomized evaluation was implemented due to excess demand for available slots:

• Lottery Design: The first 100 eligible registrants were admitted outside randomization; from the next 400, 200 were randomly assigned to QCHALLenge and 200 to control. Stratification was performed on track choice, age, residence type, and employment status.
• Sample Sizes: Analysis included 183 treated and 225 controls at 4 months, 166 treated and 218 controls at 12 months. Missing LinkedIn profiles (17 treated, 7 controls) were addressed via worst/best-case bounds.
• Mentoring Parallel Experiment: In a fall 2021 cohort, Dare IT mentors short-listed two candidates per pair, with random assignment to Mentoring or control; 152 mentor-pairs completed, with 468 further qualified, but untreated, applicants tracked.

3. Outcome Measures and Quantitative Results

The primary endpoint was commencement of a new technology-sector job post-program, defined as an active paid role in either pure-tech firms or IT roles in non-tech organizations.

• Measurement windows:
  • New job (any sector) within 4 months
  • Tech job within 4 months
  • Tech job within 12 months
• Average Treatment Effect (ATT) Calculation:

  $\mathrm{ATT}^P = P(Y=1 \mid W^P=1) - P(Y=1 \mid W^P=0)$</p> <p>Where$W^P$indicates assignment to program$P$and$Y=1$$signals job attainment.</li> <li><strong>Key Numerical Outcomes</strong>:</li> </ul> <div class='overflow-x-auto max-w-full my-4'><table class='table border-collapse w-full' style='table-layout: fixed'><thead><tr> <th>Program</th> <th>New Job 4 mo</th> <th>Tech Job 4 mo</th> <th>Tech Job 12 mo (Hazard Ratio)</th> </tr> </thead><tbody><tr> <td>Mentoring</td> <td>0.045 (0.05)</td> <td>0.129 (0.05)</td> <td>1.29 (95</tr> <tr> <td>QCHALLenge</td> <td>0.040 (0.05)</td> <td>0.089 (0.04)</td> <td>1.45 (95</tr> </tbody></table></div> <ul> <li><strong>Control Group Baselines</strong>: <ul> <li>Mentoring:$$P(Y=1)=0.293$</li> <li>QCHALLenge:$P(Y=1)=0.196$$</li> </ul></li> <li><strong><a href="https://www.emergentmind.com/topics/attentional-aggregation-att" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">ATT</a> Expressed as Percent Improvement</strong>: <ul> <li>Mentoring: 44<li>QCHALLenge: 45</ul></li> <li><strong>Robustness</strong>: Bounds following Lee and Weidner (2021) confirmed gains were significant and robust to missing outcomes.</li> </ul> <h2 class='paper-heading' id='prioritization-and-resource-allocation-under-capacity-constraints'>4. Prioritization and Resource Allocation Under Capacity Constraints</h2> <p>With demand outstripping program capacity, QCHALLenge incorporated algorithmic allocation of intervention slots to maximize aggregate employment impact:</p> <ul> <li><strong>Treatment Effect Estimation</strong>: Individualized effects ($$\tau_i^M, \tau_i^C$$) for Mentoring and QCHALLenge were estimated via cross-fitted Cox models.</li> <li><strong>Integer Programming for Allocation</strong>:</p> <p>$$\max_z \sum_i(z_{iM} \tau_i^M + z_{iC} \tau_i^C)$</p> <p>Subject to capacity constraints$Q^M, Q^C$and exclusivity$z_{iM} + z_{iC} \leq 1$$.</li> <li><strong>Optimization Outcomes</strong>: <ul> <li>Under actual quotas (28<li>Even as overall capacity expands, optimized targeting confers persistent incremental gains.</li> </ul></li> </ul> <div class='overflow-x-auto max-w-full my-4'><table class='table border-collapse w-full' style='table-layout: fixed'><thead><tr> <th>Group (Capacity)</th> <th>ATT (Optimal)</th> <th>ATT (Mentoring)</th> <th>ATT (Challenges)</th> </tr> </thead><tbody><tr> <td>Mentoring (15<td>0.199</td> <td>0.199</td> <td>0.276</td> </tr> <tr> <td>Challenges (50<td>0.244</td> <td>0.109</td> <td>0.244</td> </tr> <tr> <td>Out-group (35<td>—</td> <td>0.101</td> <td>0.073</td> </tr> </tbody></table></div> <ul> <li><strong>Mentor Selection Rules</strong>: Mentors typically selected candidates with the highest baseline chance of employment, yielding only modest incremental gain. Counterfactual allocation to the highest predicted treatment effects (“Highest CATE” policy) would have raised impact from 9.6 ppt (observed) to 17.8 ppt.</li> </ul> <h2 class='paper-heading' id='cost-effectiveness-scalability-and-generalizability'>5. Cost-Effectiveness, Scalability, and Generalizability</h2> <ul> <li><strong>Unit Costs</strong>: QCHALLenge achieves 9 ppt lift in technology sector employment at a cost of$$15 per participant, significantly less than traditional sectoral training, which often incurs costs in thousands per hire. Mentoring, with elevated staff expense, generates a 13 ppt increase in employment probability.

• Scalability: The online, peer- and expert-review model overcomes bottlenecks inherent to mentor-dependent interventions. Cohort size scalable to 500–800 without per-capita cost inflation.
• Generalizability: The framework—low-cost, authentic project-based portfolios and structured feedback—extends to parallel tracks (manual testing, automated testing, cloud, UI design) and can be adapted to other domains contingent on industry-partnered assignment design.

6. Policy Recommendations and Implications

Randomized evaluation and observational counterfactual analysis identify actionable rules for program operators:

• Individualized Treatment Effect Estimation: Pre-admission surveys and résumé features suffice for prediction.
• Program Slot Allocation: Solve the integer program above to assign quotas across Mentoring, QCHALLenge, and neither.
• Mentor Shortlisting Optimization: Select volunteers’ candidates not by “most promising,” but by “highest upside”—maximizing incremental program impact.
• Scalable Expansion: Cohort-based peer review and portfolio assessment enable effective interventions at scale.

A plausible implication is that program operators in diverse geographies and specialties can achieve maximal impact by combining randomized evaluation-driven targeting algorithms with robust peer/case-based curricula and structured feedback loops.

---

Reference: (Athey et al., 2022)