MOOClet Framework for Adaptive Digital Learning

• MOOClet Framework is a modular approach that integrates randomized A/B/n experiments with adaptive personalization for digital learning.
• It comprises a Version Manager, User Variable Store, and Policy Engine to execute experimental and personalized content assignments in real time.
• The framework enables streamlined instructor–researcher collaboration and continuous improvement through automated matching and rigorous data logging.

The MOOClet framework is a formal and modular approach for embedding randomized experimentation and adaptive personalization into the components of digital learning platforms. Developed to enable both instructors and researchers to optimize learner experiences while conducting causal inference at scale, the MOOClet formalism prescribes a software and experimental architecture wherein modular content units—MOOClets—can support seamless transitions between A/B/n randomized experiments, contextual-bandit-based personalization, and continuous improvement. The framework has been established and refined by Williams et al. (Williams et al., 2015) and Williams & Heffernan (Williams et al., 2015).

1. Formal Definition and Mathematical Structure

A MOOClet is formally defined as a tuple $M = (V, U, P)$, where:

• $V = \{v_1, v_2, \ldots, v_K\}$ is a finite set of alternate versions (arms) of a digital resource component—such as an exercise, message, email, or video segment.
• $U$ (the User Variable Store) records for each learner a vector $X \in \mathcal{X}$ of features (demographics, performance metrics, engagement history).
• $P$ designates the selection policy mapping user variables to a version in $V$. For each arriving learner with features $X = x$, the MOOClet draws $v \in V$ according to

$P(V = v \mid X = x) = p_v(x)$

with $\sum_{v \in V} p_v(x) = 1$.

Key statistical quantities in MOOClet experiments and personalization include:

• The marginal expected outcome for version $v$, $E[Y \mid V=v] = \mu_v$ (where $Y$ is the observed reward/outcome).
• The conditional expected outcome for version $v$ given learner features, $\mu_v(x) = E[Y \mid X = x, V = v]$.
• The optimal personalized version, $v^*(x) = \arg\max_{v\in V} \mu_v(x)$.

Uniform randomization $p_v(x) = 1/|V|$ for all $v, x$ yields classical A/B or multi-arm trials, whereas $p_v(x) = 1$ for $v = v^*(x)$ recovers greedy personalization. Intermediate policies (e.g., $\epsilon$-greedy, UCB, Thompson Sampling) fit the same schema (Williams et al., 2015, Williams et al., 2015).

2. Core Software Architecture

The MOOClet framework prescribes three cooperating architectural modules within a learning platform:

• Version Manager: Interface for authoring and storing modular content versions, keyed per MOOClet instance.
• User Variable Store (UVS): Records, aggregates, and provides access (with privacy constraints) to learner IDs, feature vectors $X$, event logs, assigned versions $V$, and observed outcomes $Y$.
• Policy Engine: Realizes the assignment rule $p_v(x)$, supporting static randomization, general contextual mappings (e.g., $p_v(x) = f_\theta(x)$), and adaptive algorithms informed by accumulated UVS statistics.

This architecture exposes an API endpoint, such as getVersion(userId), and enables policy code upload (e.g., Python, SQL-UDF) to operationalize assignment and adaptation logic (Williams et al., 2015, Williams et al., 2015).

3. Unified Perspective: Experimentation and Personalization

The formalism establishes that A/B/n experimentation and adaptive personalization are special cases of version assignment conditioned on a variable—either an experimental (random) variable or a deterministically derived user-characteristic. In both cases, the software layer executes:

$\text{Version}^i = f(Z^i)$

where $Z^i$ is (a) randomly assigned for pure experimentation, or (b) derived from $X^i$ for personalization.

This unified view allows seamless transitions between:

• Classical randomized experiments (policy fixed, uniformly random $Z_{\text{exp}}$).
• Contextual or subgroup personalization (policy $P$ maps $X$ to version deterministically or probabilistically).
• Continuous personalization (updating both $U$ and $P$ in real time) (Williams et al., 2015, Williams et al., 2015).

4. Adaptive Algorithms for Experimentation and Personalization

The MOOClet policy engine can instantiate a variety of assignment rules, supporting a methodological continuum:

• Static A/B/n Testing: $p_v(x) = 1/|V|$; use sample mean rewards for per-arm inference. After $n_v$ observations on each version $v$, estimate $\hat\mu_v$ and test hypotheses (e.g., $H_0\!:\! \mu_{v_1} = \mu_{v_2}$).
• Contextual Bandits: Employs learner features to condition version probabilities. For instance, UCB and Thompson Sampling algorithms are used to maximize reward while balancing exploration and exploitation.

UCB pseudocode within the MOOClet system (for each arm $v$ as assigned to $t$-th learner):

for each v in V: if n_v == 0: U_v = float('inf') else: U_v = (sum_v / n_v) + c * sqrt(log(t) / n_v) v_t = argmax_v U_v

In contextual bandits, policy probabilities may be specified as:

$$p_{j, c} = \frac{\exp[\beta \hat{\mu}_{j, c}]}{\sum_{\ell}\exp[\beta \hat{\mu}_{\ell, c}]}$$

for context $c$ (Williams et al., 2015, Williams et al., 2015).

5. Instructor–Researcher Collaboration Workflow

The MOOClet framework encodes a structured workflow for aligning instructional needs with research opportunities:

1. Instructor Specification: Details component(s) modifiable, feasible version count, permissible learner features $X$, and outcome metrics $Y$.
2. Researcher Specification: Describes experimental contrasts, target hypotheses/personalization aims, and preferred assignment algorithm(s).
3. Automated Matching: A compatibility score $C_{ij}$ is computed for each instructor–researcher pair, aggregating weights for modifiable components, available features, and outcome alignment:

$$C_{ij} = w_1 \cdot \text{match}(\text{components}_i, \text{needed}_j) + w_2 \cdot \text{match}(\text{features}_i, \text{features}_j) + w_3 \cdot \text{match}(\text{outcomes}_i, \text{outcomes}_j)$$

Pairing is performed via a greedy or maximum-weight bipartite matching. After a pairing is established, the MOOClet platform auto-deploys experiment/personalization code in the instructor's course component (Williams et al., 2015).

6. Data Schema and Instrumentation

Every MOOClet instance logs the following schema per learner interaction:

Field name	Type	Description
learner_id	string/int	Pseudonymized user identifier
timestamp	datetime	UTC time of rendering or event
M_id	string	Unique ID of the MOOClet
version_id	string	Indices version $v\in V$ served
X_1, X_2, …	various	Learner/context features as available
event_type	enum	“render”, “submit”, “click”, etc.
Y	float/int	Outcome measurement (e.g., score, click indicator)

Researchers and instructors can query slices of this log for analysis or real-time updates to assignment policy (Williams et al., 2015).

7. Practical Examples and Implementation Patterns

Concrete use cases of the MOOClet framework include:

• Welcome-Email MOOClet: Two email variants (friendly vs. data-driven). Static random assignment, outcome $Y$ as click-through, then switching to personalization upon observed significance.
• Reflection Exercise MOOClet: Three content prompts; features include a pre-test score with outcome as self-reported understanding. Pilot data supports fitting a linear model $\mu_v(x) = \alpha_v + \beta_v x$. Assignment transitions from random to $v^*(x) = \arg\max_v (\alpha_v + \beta_v x)$ as model reliability grows.

Best practices documented include: starting with a limited number of versions for statistical power, predefining ethical data collection, revealing all collected variables to instructors, iterating rapidly across components, and initial use of pure randomization before adaptive deployment (Williams et al., 2015).

• Williams, J. J., Kim, J., Rafferty, A., Maldonado, S., Gajos, K., Lasecki, W. S., & Heffernan, N. T. (2015). "Supporting Instructors in Collaborating with Researchers using MOOClets" (Williams et al., 2015).
• Williams, J. J. & Heffernan, N. (2015). "A Methodology for Discovering how to Adaptively Personalize to Users using Experimental Comparisons" (Williams et al., 2015).