All Posts | Emily Riederer

Role-Based Access Control for Quarto sites with Netlify Identity

quarto

rmarkdown

workflow

A quick tech note on Netlify’s managed authentication solution

Crosspost: Data discovery doesn’t belong in ad hoc queries

data

workflow

elt

crosspost

Data teams may struggle to quantify the benefits of good data documentation. But running countless ad hoc validation queries can incur both computational and cognitive cost.

Base Python Rgonomic Patterns

rstats

python

tutorial

Getting comfortable in a new language is more than the packages you use. Syntactic sugar in base python increases the efficiency, and aesthetics of python code in ways that R users may enjoy in packages like glue and purrr. This post collects a miscellaneous grab bag of tools for wrangling, formatting (f-strings), repeating (list comprehensions), faking data, and saving objects (pickle)

Crosspost: Why You Need Data Documentation in 2024

data

workflow

elt

crosspost

Data documentation isn’t a box to check; it’s an active member of your team with many jobs-to-be-done. In this cross-post with Select Star, I write about how effective documentation can be your data products’ developer advocate for users, project manager for developers, and chief of staff for data leaders

`polars`’ Rgonomic Patterns

rstats

python

tutorial

In this follow-up post to Python Rgonomics, we deep dive into some of the advanced data wrangling functionality in python’s polars package to see how it’s powertools like column selectors and nested data structures mirror the best of dplyr and tidyr’s expressive and concise syntax

Crosspost: Why you’re closer to data documentation than you think

data

workflow

elt

crosspost

Writing is thinking; documenting is planning and executing. In this cross-post with Select Star, I write about how teams can produce high-quality and maintainble documentation by smartly structuring planning and development documentation and effeciently recycling them into long-term, user-friendly docs

Python Rgonomics

rstats

python

Switching languages is about switching mindsets - not just syntax. New developments in python data science toolings, like polars and seaborn’s object interface, can capture the ‘feel’ that converts from R/tidyverse love while opening the door to truly pythonic workflows

Big ideas from the 2023 Causal Data Science Meeting

causal

Five highlights and links to select talks

Industry information management for causal inference

causal

data

Proactive collection of data to comply or confront assumptions

Crosspost: The Art of Abstraction in ETL

data

workflow

elt

crosspost

Rounding out my three-part ETL series form Airbyte’s developer blog

The Art of Abstraction in ETL: Dodging Data Extraction Errors

data

workflow

elt

crosspost

Cross-post from guest post on Airbyte’s developer blog

Crosspost: Power up your data quality with grouped checks

data

dbt

crosspost

After a prior post on the merits of grouped data quality checks, I demo my newly merged implementation for dbt

Goin’ to Carolina in my mind (or on my hard drive)

data

sql

Out-of-memory processing of North Carolina’s voter file with DuckDB and Apache Arrow

Oh, I’m sure it’s probably nothing

rstats

python

sql

data

data-disasters

How we do (or don’t) think about null values and why the polyglot push makes it all the more important

Update: grouped data quality check PR merged to dbt-utils

data

dbt

After a prior post on the merits of grouped data quality checks, I demo my newly merged implementation for dbt

Using databases with Shiny

rstats

shiny

data

Key issues when adding persistent storage to a Shiny application, featuring {golem} app development and Digital Ocean serving

How to Make R Markdown Snow

rstats

rmarkdown

Much like ice sculpting, applying powertools to absolutely frivolous pursuits

Make grouping a first-class citizen in data quality checks

data

Which of these numbers doesn’t belong? -1, 0, 1, NA. You can’t judge data quality without data context, so our tools should enable as much context as possible.

Why machine learning hates vegetables

data-disasters

A personal encounter with ‘intelligent’ data products gone wrong

Update: column-name contracts with dbtplyr

rstats

dbt

Following up on ‘Embedding Column-Name Contracts… with dbt’ to demo my new dbtplyr package to further streamline the process

A lightweight data validation ecosystem with R, GitHub, and Slack

rstats

data

elt

A right-sized solution to automated data monitoring, alerting, and reporting using R (pointblank, projmgr), GitHub (Actions, Pages, issues), and Slack

Workflows for querying databases via R

rstats

workflow

sql

Tricks for modularizing and refactoring your projects SQL/R interface. (Image source techdaily.ca)

Understanding the data (error) generating processes for data validation

data

elt

A data consumer’s guide to validating data based on the failure modes data producer’s try to avoid

A Tale of Six States: Flexible data extraction with scraping and browser automation

data

elt

python

Exploring how Playwright‘s headless browser automation (and its friends) can help unite the states’ data

Embedding column-name contracts in data pipelines with dbt

data

sql

dbt

dbt supercharges SQL with Jinja templating, macros, and testing – all of which can be customized to enforce controlled vocabularies and their implied contracts on a data model

Causal design patterns for data analysts

causal

data

tutorial

An informal primer to causal analysis designs and data structures

Resource Round-Up: Causal Inference

causal

resources

Free books, lectures, blogs, papers, and more for a causal inference crash course

Building a team of internal R packages

rstats

pkgdev

jtbd

On the jobs-to-be-done and design principles for internal tools

Generating SQL with {dbplyr} and sqlfluff

rstats

data

sql

tutorial

Using the tidyverse’s expressive data wrangling vocabulary as a preprocessor for elegant SQL scripts. (Image source techdaily.ca)

Introducing the {convo} package

rstats

pkgdev

data

An R package for maintaining controlled vocabularies to encode contracts between data producers and consumers

Sticker-driven maintenance

workflow

Marketing maintenance work with irrational exuberance

crosstalk: Dynamic filtering for R Markdown

rstats

tutorial

An introduction to browser-based interactivity of htmlwidgets – no Shiny server required!

Column Names as Contracts

data

workflow

rstats

Using controlled dictionaries for low-touch documentation, validation, and usability of tabular data

A beginner’s guide to Shiny modules

shiny

rstats

workflow

tutorial

Don’t believe the documentation! Shiny modules aren’t just for advanced users; they might just be a great entry point for development

Resource Round-Up: Latent and Lasting Documentation

resources

rstats

Readings and assorted ideas about creating and maintaining low-overhead documentation

RMarkdown CSS Selector Tips

rmarkdown

rstats

A few tips and tools for finding the right selectors to style in RMarkdown

projmgr: Managing the human dependencies of your projects

rstats

workflow

A walkthrough of using the projmgr package for GitHub-based project management via R

RMarkdown Driven Development: the Technical Appendix

pkgdev

workflow

rstats

rmarkdown

A recommended tech stack for implementing RMarkdown Driven Development

Resource Round-Up: R in Industry Edition

resources

rstats

Case studies of the impact of R use on organizational culture and collaboration

Resource Round-Up: Reproducible Research Edition

resources

workflow

An annotated bibliography of advice for getting started with reproducible research

Rtistic: A package-by-numbers repo

rstats

pkgdev

A walkthrough of a GitHub template for making your own RMarkdown and ggplot2 theme package

Notes on supporting conference speakers

speaking

notes

Conference planning tips to design a good speakers experience

RMarkdown Driven Development (RmdDD)

pkgdev

rmarkdown

rstats

workflow

A workflow for refactoring one-time analyses to sustainable data products

Notes on preparing a tech talk

speaking

notes

A proposed workflow for methodically developing a good presentations