Welcome!

Posts

Crosspost: Data discovery doesn’t belong in ad hoc queries

data
workflow
elt
crosspost
Data teams may struggle to quantify the benefits of good data documentation. But running countless ad hoc validation queries can incur both computational and cognitive cost.
2024-07-18

Base Python Rgonomic Patterns

rstats
python
tutorial
Getting comfortable in a new language is more than the packages you use. Syntactic sugar in base python increases the efficiency, and aesthetics of python code in ways that R users may enjoy in packages like glue and purrr. This post collects a miscellaneous grab bag of tools for wrangling, formatting (f-strings), repeating (list comprehensions), faking data, and saving objects (pickle)
2024-01-20

Crosspost: Why You Need Data Documentation in 2024

data
workflow
elt
crosspost
Data documentation isn’t a box to check; it’s an active member of your team with many jobs-to-be-done. In this cross-post with Select Star, I write about how effective documentation can be your data products’ developer advocate for users, project manager for developers, and chief of staff for data leaders
2024-01-15

polars’ Rgonomic Patterns

rstats
python
tutorial
In this follow-up post to Python Rgonomics, we deep dive into some of the advanced data wrangling functionality in python’s polars package to see how it’s powertools like column selectors and nested data structures mirror the best of dplyr and tidyr’s expressive and concise syntax
2024-01-13

Crosspost: Why you’re closer to data documentation than you think

data
workflow
elt
crosspost
Writing is thinking; documenting is planning and executing. In this cross-post with Select Star, I write about how teams can produce high-quality and maintainble documentation by smartly structuring planning and development documentation and effeciently recycling them into long-term, user-friendly docs
2024-01-05

Python Rgonomics

rstats
python
Switching languages is about switching mindsets - not just syntax. New developments in python data science toolings, like polars and seaborn’s object interface, can capture the ‘feel’ that converts from R/tidyverse love while opening the door to truly pythonic workflows
2023-12-30

Big ideas from the 2023 Causal Data Science Meeting

causal
Five highlights and links to select talks
2023-11-18

Industry information management for causal inference

causal
data
Proactive collection of data to comply or confront assumptions
2023-05-30

Crosspost: The Art of Abstraction in ETL

data
workflow
elt
crosspost
Rounding out my three-part ETL series form Airbyte’s developer blog
2023-05-03

The Art of Abstraction in ETL: Dodging Data Extraction Errors

data
workflow
elt
crosspost
Cross-post from guest post on Airbyte’s developer blog
2023-03-22

Crosspost: Power up your data quality with grouped checks

data
dbt
crosspost
After a prior post on the merits of grouped data quality checks, I demo my newly merged implementation for dbt
2023-01-17

Goin’ to Carolina in my mind (or on my hard drive)

data
sql
Out-of-memory processing of North Carolina’s voter file with DuckDB and Apache Arrow
2022-09-25

Oh, I’m sure it’s probably nothing

rstats
python
sql
data
data-disasters
How we do (or don’t) think about null values and why the polyglot push makes it all the more important
2022-09-05

Update: grouped data quality check PR merged to dbt-utils

data
dbt
After a prior post on the merits of grouped data quality checks, I demo my newly merged implementation for dbt
2022-08-26

Using databases with Shiny

rstats
shiny
data
Key issues when adding persistent storage to a Shiny application, featuring {golem} app development and Digital Ocean serving
2022-01-02

How to Make R Markdown Snow

rstats
rmarkdown
Much like ice sculpting, applying powertools to absolutely frivolous pursuits
2021-12-11

Make grouping a first-class citizen in data quality checks

data
Which of these numbers doesn’t belong? -1, 0, 1, NA. You can’t judge data quality without data context, so our tools should enable as much context as possible.
2021-11-27

Why machine learning hates vegetables

data-disasters
A personal encounter with ‘intelligent’ data products gone wrong
2021-11-10

Update: column-name contracts with dbtplyr

rstats
dbt
Following up on ‘Embedding Column-Name Contracts… with dbt’ to demo my new dbtplyr package to further streamline the process
2021-09-21

A lightweight data validation ecosystem with R, GitHub, and Slack

rstats
data
elt
A right-sized solution to automated data monitoring, alerting, and reporting using R (pointblank, projmgr), GitHub (Actions, Pages, issues), and Slack
2021-08-26

Workflows for querying databases via R

rstats
workflow
sql
Tricks for modularizing and refactoring your projects SQL/R interface. (Image source techdaily.ca)
2021-07-14

Understanding the data (error) generating processes for data validation

data
elt
A data consumer’s guide to validating data based on the failure modes data producer’s try to avoid
2021-05-27

A Tale of Six States: Flexible data extraction with scraping and browser automation

data
elt
python
Exploring how Playwright‘s headless browser automation (and its friends) can help unite the states’ data
2021-05-08

Embedding column-name contracts in data pipelines with dbt

data
sql
dbt
dbt supercharges SQL with Jinja templating, macros, and testing – all of which can be customized to enforce controlled vocabularies and their implied contracts on a data model
2021-02-06

Causal design patterns for data analysts

causal
data
tutorial
An informal primer to causal analysis designs and data structures
2021-01-30

Resource Round-Up: Causal Inference

causal
resources
Free books, lectures, blogs, papers, and more for a causal inference crash course
2021-01-30

Building a team of internal R packages

rstats
pkgdev
jtbd
On the jobs-to-be-done and design principles for internal tools
2021-01-21

Generating SQL with {dbplyr} and sqlfluff

rstats
data
sql
tutorial
Using the tidyverse’s expressive data wrangling vocabulary as a preprocessor for elegant SQL scripts. (Image source techdaily.ca)
2021-01-16

Introducing the {convo} package

rstats
pkgdev
data
An R package for maintaining controlled vocabularies to encode contracts between data producers and consumers
2020-12-30

Sticker-driven maintenance

workflow
Marketing maintenance work with irrational exuberance
2020-09-20

crosstalk: Dynamic filtering for R Markdown

rstats
tutorial
An introduction to browser-based interactivity of htmlwidgets – no Shiny server required!
2020-09-12

Column Names as Contracts

data
workflow
rstats
Using controlled dictionaries for low-touch documentation, validation, and usability of tabular data
2020-09-06

A beginner’s guide to Shiny modules

shiny
rstats
workflow
tutorial
Don’t believe the documentation! Shiny modules aren’t just for advanced users; they might just be a great entry point for development
2020-07-26

Resource Round-Up: Latent and Lasting Documentation

resources
rstats
Readings and assorted ideas about creating and maintaining low-overhead documentation
2020-07-03

RMarkdown CSS Selector Tips

rmarkdown
rstats
A few tips and tools for finding the right selectors to style in RMarkdown
2020-06-30

projmgr: Managing the human dependencies of your projects

rstats
workflow
A walkthrough of using the projmgr package for GitHub-based project management via R
2020-05-14

RMarkdown Driven Development: the Technical Appendix

pkgdev
workflow
rstats
rmarkdown
A recommended tech stack for implementing RMarkdown Driven Development
2020-02-01

Resource Round-Up: R in Industry Edition

resources
rstats
Case studies of the impact of R use on organizational culture and collaboration
2019-08-30

Resource Round-Up: Reproducible Research Edition

resources
workflow
An annotated bibliography of advice for getting started with reproducible research
2019-08-30

Rtistic: A package-by-numbers repo

rstats
pkgdev
A walkthrough of a GitHub template for making your own RMarkdown and ggplot2 theme package
2019-05-25

Notes on supporting conference speakers

speaking
notes
Conference planning tips to design a good speakers experience
2019-05-07

RMarkdown Driven Development (RmdDD)

pkgdev
rmarkdown
rstats
workflow
A workflow for refactoring one-time analyses to sustainable data products
2019-05-04

Notes on preparing a tech talk

speaking
notes
A proposed workflow for methodically developing a good presentations
2019-04-20
No matching items

Go to All Posts →

Talks

Python Rgonomics

workflow
python
rstats
A survey of modern python tooling that “feels good” to R users
Aug 15, 2024

Data Downtime Horror Stories Panel

data
Panel discussion with Chad Sanderson and Joe Reis, hosted by Monte Carlo Data, on our thorniest brushes with data downtime, leading data teams to tackle data quality at scale with testing, contracts, observability and monitoring, and more.
Oct 23, 2023

Operationalizing Column-Name Contracts with dbtplyr

workflow
rmarkdown
rstats
An exploration of how data producers and consumers can use column names as interfaces, configuations, and code to improve data quality and discoverability. The second half of the talk demonstrates how to implement these ideas with my dbtplyr dbt package.
Sep 21, 2023

Scaling Personalized Volunteer Emails

data
elt
An overview of the data stack used to automate over 50,000 personalized emails to voter turnout volunteers using BigQuery, dbt, Census, and MailChimp
Jun 21, 2023

Causal Design Patterns

causal
An overview of basic research design patterns in causal inference, modern extensions, and data management strategies to set up a causal inference initiative for success
Jun 7, 2023

DataFold Data Quality Meet Up

elt
data
Joined a panel of speakers to discuss tips and tricks for running dbt at scale
May 12, 2023

Posit Data Science Hangout

rstats
workflow
Each week, host Rachael Dempsey invites an accomplished data science leader to talk about their experience and answer questions from the audience. The discussion focuses mainly on the human elements of data science leadership. There’s no sales or marketing fluff, just great insights from inspiring professionals.
Apr 13, 2023

Evaluation without Experimentation

causal
An introduction to inverse propensity of treatment weighting for program evaluation with applications to Two Million Texans’ relational organizing campaign during the 2022 midterms
Mar 22, 2023

Taking Flight with Shiny: a Modules-First Approach

workflow
rstats
shiny
An argument for the individual and organization-wide benefits of teaching new developers Shiny with a modules-first paradigm.
Mar 15, 2023

The Data (error) Generating Process

data
Interrogating the data generating process to devise better data quality tests.
Nov 12, 2022

The Data Engineering Podcast: Column Names as Contracts

data
Discussing how column names can serve as a light-weight alternative to data catalogs and contracts and how to implement this approach with dbtplyr
Jan 12, 2022

UIUC STAT447 (Data Science Programming) Guest Lecture

rstats
workflow
Discussing how to move from scripting to tool development, designing tools in enterprise, and navigating diverse data career paths
Nov 17, 2021

Column Names as Contracts

data
Exploring the benefits of using controlled vocabularies to encode metadata in column names, and demonstrations of implementing this approach with the convo R package or dbt extensions of SQL.
Feb 26, 2021

oRganization: Design patterns for internal packages

workflow
pkgdev
rstats
An overview of the unique design challenges and opportunities when building R packages for use inside of a single organization versus open-source. By using the jobs-to-be-done framework, this talk explores how internal packages can be better teammates by following specific design patterns for API design, testing, documentaiton, and more.
Jan 21, 2021

projmgr: Managing the human dependencies of your project

workflow
rstats
pkgdev
A lightning talk on key features of the projmgr package which brings enables code-based planning and reporting workflows grounded in GitHub issues and milestones
Jul 6, 2020

RMarkdown Driven Development

workflow
rmarkdown
rstats
How and why to refactor one time analyses in RMarkdown into sustainable data products
Jan 30, 2020

Assorted talks on designing analytical tools and communities for enterprise

workflow
rmarkdown
rstats
A variety of related talks to creating innersource culture with R packages and related tools
Nov 1, 2017

tidycf: Turning analysis on its head by turning cashflows on their side

workflow
pkgdev
rstats
A case study on building an internal R package for customer lifetime value modeling at Capital One and leading broader analyst adoption of open-source tooling and reproducible workflows through a community of practice.
Nov 1, 2017
No matching items

Go to All Talks →

Publications

Go to All Publications →