Start with the improvement
A learning curve is a more informative starting point than raw data volume because it asks what changes as experience accumulates. A dataset can be large without improving a task customers value. The moat question adds another requirement: can the company keep producing a useful improvement that rivals cannot readily reproduce? Both the benefit and the barrier need evidence beyond the number of stored records.
Define the outcome before examining the collection. A support system might target correct resolutions under a fixed review policy; an industrial system might target a defined task success rate under stated conditions. These are illustrative evaluation choices. The right metric depends on the job. Explain what the metric misses, which errors are costly and whether a gain transfers to the customer's real operating environment.
Then separate three questions. Does new experience improve the system? Do customers benefit from the improvement? Does the company possess a defensible position in obtaining or applying that experience? A positive answer to one does not settle the others. A capable model can be commercially interchangeable; an exclusive archive can be unhelpful; a useful feedback loop can still be available to every competitor.
Examine how the learning works
Hagiu and Wright model firms learning either across customers or through repeated use by the same customer. Their analysis makes competitive advantage depend on the shape of learning functions, asymmetries and accumulated data, among other conditions. This is theory, not evidence that a particular startup has a data network effect. It explains why the phrase more users means more data leaves important strategic questions unanswered.
Use that distinction to design an evaluation. If learning is within one account, test whether that account's history improves its own outcomes. If learning is across accounts, test whether permissible experience from one group improves results for a separate group. Do not describe customer data as pooled unless the rights and actual process support that statement. A claimed effect should have an identifiable path from experience to benefit.
Plot performance against meaningful additions to experience while keeping the evaluation protocol stable. The curve might flatten, shift after a model change or improve only for familiar cases. Report those differences. A flat section can mean collection is no longer the bottleneck; a sudden gain may come from a better model rather than exclusive data. The strategic explanation should follow the result instead of attributing all progress to a dataset.
Test diversity and transfer
Lin and colleagues studied data scaling for robotic manipulation and found environment and object diversity more important than simply adding demonstrations once a threshold was reached. The result concerns their experimental tasks and setting. It does not establish a universal rule for every robot or dataset. It is a useful example of why counting observations without describing their diversity can conceal how much useful experience was collected.
For a company, separate repetition from coverage. Record which environments, conditions and failure cases are represented, and hold out meaningful cases for evaluation. A larger collection that repeats the same easy situation may add little information. Conversely, a carefully selected correction may reveal a weakness the system rarely encountered. Compare both approaches rather than assuming that collection scale and learning value move together.
Keep the summary understandable to the customer. State what was withheld, how success was defined and whether the evaluation included human intervention. Preserve the distinction between controlled performance and production outcomes. These steps are proposed evidence practices; the cited robotics study does not validate a commercial moat. Firm-level advantage still requires proof of useful improvement, defensible access and a credible comparison with alternatives.
Compare against what a rival can use
A proprietary-data claim needs a realistic alternative. Open X-Embodiment reports transfer from pooled robotics datasets across platforms, illustrating that relevant learning can come from shared resources. A company should identify the public and commercially available inputs a capable rival can use. The baseline should be competitive enough to test the claimed exclusivity, with licences and permitted use checked for the comparison being made.
Imitation also needs task-level evaluation. Gudibande and colleagues found that their language-model imitation systems could resemble a stronger model's style while failing to close capability gaps on tasks poorly represented in imitation data. The work is a 2023 study with a defined model setting. It supports distinguishing appearance from demonstrated capability; it does not establish that current imitation methods have a permanent ceiling.
For a founder, compare the production system with a permissible public baseline, an appropriate commercial alternative and a version without the claimed private input. Document differences in compute, model choice and human review. If those conditions cannot be matched, describe the limitation. A clean explanation of uncertainty is more useful than attributing an unmatched system's whole performance gap to exclusive data.
Document the whole feedback loop
A defensible learning-loop hypothesis names the source of observations, the right to use them, the evaluation step and the route back into a better product. It should also describe what happens when a customer leaves or withdraws permission. Access, ownership and allowed use are separate questions. Ask qualified advisers to examine the relevant agreements rather than inferring rights from the fact that data passed through a system.
The improvement process itself deserves inspection. Who reviews corrections? How are errors detected? Can an apparent gain be reproduced on a later held-out set? What happens when the operating environment changes? These questions make the loop testable without requiring public disclosure of sensitive records. Protecting the collection is useful only if the permitted process can keep extracting relevant learning from it.
Today's Moat Index considers data claims through the relevant Powers in its v5.0 method, which values measured evidence above claims. Broader treatment of learning mechanisms and graph-backed recipes is coming. Bring a measured curve, a rights explanation and a rival comparison to a review. The question is not how impressive the archive sounds. It is which supported advantage persists when competitors receive better tools and public starting points.
Sources
Sources support the linked claims; our interpretation and checklists establish no outcome for your company.