Operationalizing Column-Name Contracts with dbtplyr
dbtplyr dbt package.
url_video: “”
At Coalesce for dbt user audience:
At posit::conf for R user audience:
Slides
Video - posit::conf for R User Audience coming soon!
Post - Column Name Contracts
Post - Column Name Contracts in dbt
Post - Column Name Contracts with dbtplyr
Complex software systems make performance guarantees through documentation and unit tests, and they communicate these to users with conscientious interface design.
However, published data tables exist in a gray area; they are static enough not to be considered a “service” or “software”, yet too raw to earn attentive user interface design. This ambiguity creates a disconnect between data producers and consumers and poses a risk for analytical correctness and reproducibility.
In this talk, I will explain how controlled vocabularies can be used to form contracts between data producers and data consumers. Explicitly embedding meaning in each component of variable names is a low-tech and low-friction approach which builds a shared understanding of how each field in the dataset is intended to work.
Doing so can offload the burden of data producers by facilitating automated data validation and metadata management. At the same time, data consumers benefit by a reduction in the cognitive load to remember names, a deeper understanding of variable encoding, and opportunities to more efficiently analyze the resulting dataset. After discussing the theory of controlled vocabulary column-naming and related workflows, I will illustrate these ideas with a demonstration of the {dbtplyr} dbt package which helps analytics engineers get the most value from controlled vocabularies by making it easier to effectively exploit column naming structures while coding.
Coming Soon!