Meta tags:
author=Scott Czepiel;
description=work, data, and working_with_data;

Headings (most frequently used words):

to, and, variables, observations, your, how, analysis, for, this, time, is, no, different, obfuscating, primary, keys, load, ipums, datasets, into, relational, database, exploratory, with, the, 2020, american, national, election, studies, stasis, simple, static, site, generator, not, tank, perfectly, good, power, minimap, data, front, end, eda, using, window, functions, cross, joins, count, events, above, threshold, migrating, gatsby, links, topics, recent, entries,

Text of the page (most frequently used words):
the (70), and (39), this (25), data (22), for (22), with (16), that (14), analysis (13), your (11), 2020 (11), will (11), web (10), words (10), czep (10), how (9), not (9), but (9), post (8), end (7), into (7), work (6), #gatsby (6), front (6), site (6), generator (6), exploratory (6), time (6), have (6), like (6), one (6), you (6), some (6), power (5), #static (5), ipums (5), datasets (5), database (5), more (5), writing (5), apr (5), are (5), variables (4), using (4), minimap (4), election (4), about (4), blog (4), from (4), sql (4), share (4), text (4), feature (4), what (4), all (4), report (4), new (4), observations (3), window (3), cross (3), joins (3), count (3), events (3), above (3), eda (3), good (3), stasis (3), simple (3), load (3), relational (3), primary (3), different (3), page (3), just (3), along (3), way (3), out (3), over (3), still (3), need (3), actually (3), because (3), use (3), when (3), package (3), make (3), after (3), case (3), file (3), right (3), can (3), files (3), useful (3), may (3), also (3), most (3), their (3), get (3), python (3), own (3), anes (3), 2021 (3), working_with_data (2), migrating (2), functions (2), threshold (2), tank (2), perfectly (2), american (2), national (2), studies (2), obfuscating (2), keys (2), statistics (2), rss (2), history (2), next (2), migration (2), learned (2), little (2), react (2), while (2), era (2), much (2), least (2), few (2), encountered (2), feb (2), written (2), since (2), posts (2), though (2), lot (2), handy (2), today (2), back (2), myself (2), solution (2), level (2), ids (2), 700 (2), browser (2), motivations (2), had (2), between (2), thoughts (2), might (2), small (2), high (2), view (2), working (2), see (2), help (2), first (2), application (2), read (2), project (2), anyone (2), usually (2), they (2), estimate (2), variance (2), target (2), even (2), 000 (2), website (2), generate (2), content (2), things (2), problem (2), would (2), dataset (2), available (2), survey (2), voters (2), wrote (2), script (2), year (2), common (2), users (2), photos (2), human (2), reaction (2), patterns (2), find (2), future (2), markets (2), send, feedback, here, recent, entries, culture, linux, topics, feed, ancient, home, links, subscribe, via, completed, jekyll, bit, node, taking, flat, bootstrap, engineered, modern, javascript, learn, style, now, workflow, works, discuss, decided, specifics, during, 900, haven, don, ordinarily, think, candidates, boring, everyday, certainly, internalized, tricks, those, rare, moments, sat, thought, wow, query, beautiful, involved, two, function, number, occurring, each, score, generating, hoc, session, reactjs, last, outlined, lofty, ambitions, graphical, tool, easier, describing, struggling, context, switch, programming, whirlwind, tour, inspirations, start, deep, backend, then, journey, again, hope, made, crazy, thing, 100, mar, sublime, has, called, which, shows, condensed, version, hand, margin, gives, looks, zoomed, far, could, screen, navigating, around, very, large, took, inspiration, thinking, showcase, demonstrate, possible, leveraging, please, detailed, introduction, 800, fundamental, science, procedures, prerequisite, doing, testing, reviewing, analyses, junior, scientists, look, metric, trickier, aspects, process, where, spend, our, haste, easy, overlook, another, critical, piece, population, size, specifically, many, unique, experiment, assignments, expect, short, tenure, retired, replaced, risk, spending, discussing, rather, than, contributing, real, describe, frustrations, try, too, should, write, hard, exercise, engineering, skills, ways, dec, been, three, audiences, researchers, seeking, feeling, intimidated, its, massive, structure, dearth, resources, programmers, interested, general, purpose, tabulation, visualization, interest, differences, biden, trump, months, ahead, largely, tab, deck, includes, introductory, notes, smattering, approach, code, github, htad, repository, pre, 300, ten, years, ago, metadata, extracts, order, system, personally, reached, once, every, released, often, throughout, additional, supplemental, clunky, requires, needs, extra, space, uncompress, raw, gets, job, done, there, better, mainly, thanks, development, rdbms, such, ideal, storage, census, microdata, updated, method, simplifies, moving, postgresql, ipumsr, hacky, 200, follow, requests, clients, building, urls, containing, model, appealing, nothing, screams, amateurs, inviting, user, redirecting, them, sending, customer, billing, invoice, linking, photo, sharing, affects, backed, applications, surface, permalinks, accessed, key, embedded, url, anything, detail, products, orders, comments, profiles, etc, research, question, discussion, options, consider, depending, invoices, 500, yes, bored, talking, going, away, midst, major, technological, transformation, scale, equally, impactful, adoption, mobile, phones, internet, personal, computers, technology, scary, unprecedented, happens, rooted, familiar, historical, undue, alarmism, potential, impacts, skill, jobs, bullish, upcoming, eternal, optimist, snake, oil, sell, incentives, ingenuity, competitive, point, direction, must, tread, haplessly, blind, trust, sort, best, neither, avoidance, rejection, nor, fear, doomerism, provide, framework, guide, shaping, 2026,

Text of the page (random words):
variables and observations variables and observations work data and working_with_data this time is no different 14 apr 2026 czep work 4 000 words yes we are all bored of talking about it but it s not going away we are in the midst of a major technological transformation on a scale equally as impactful as the adoption of mobile phones of the internet of personal computers even though this new technology is scary and unprecedented human reaction patterns are not we will find that much of what happens next is rooted in familiar historical patterns i am writing in reaction to what i see as undue alarmism over the potential impacts of ai on high skill jobs i find myself bullish for this upcoming future not because i m an eternal optimist or have some snake oil to sell but because history incentives human ingenuity and competitive markets all point in that direction while we must not tread haplessly in a blind trust that markets will sort things out for the best neither avoidance and rejection nor fear and doomerism will provide the framework to guide us in shaping this future obfuscating your primary keys 07 may 2021 czep web 6 500 words one of the most common follow up feature requests i get from clients after building a web application is to make urls containing model ids more appealing nothing screams we are amateurs like inviting a new user to your website and redirecting them to users 4 or sending a customer a billing invoice at invoices 12 or linking to a page on your new photo sharing site at photos 31 this is a common problem that affects database backed web applications that need to surface permalinks to content that is accessed with a primary key embedded in the url this can be anything with a detail page view users products orders comments photos profiles posts etc in this post i would like to share my own research into this question with discussion of the options you might consider depending on your use case how to load ipums datasets into a relational database 23 apr 2021 czep data 3 200 words ten years ago i wrote a hacky python script to read the metadata from ipums extracts in order to load the datasets into a relational database system i ve personally reached for this script at least once a year since i wrote it every time new ipums datasets are released and often throughout the year when i need additional data for some supplemental analysis it s a little clunky requires python 2 needs a lot of extra space to uncompress the raw data files but it still gets the job done but today there is a better way mainly thanks to the ipums project with the development of the ipumsr package an rdbms such as postgresql is an ideal solution for storage of census microdata and in this post i will share an updated method that simplifies moving the data into the database exploratory analysis with the 2020 american national election studies 17 apr 2021 czep data 300 words i have been working with the 2020 anes dataset and would like to share my report on exploratory analysis with the 2020 anes pre election survey this report may be useful for three different audiences 1 researchers seeking to use the anes dataset in their work but feeling intimidated by its massive structure and the dearth of resources available on the web for using r with survey data 2 r programmers interested in general purpose tabulation and data visualization and 3 anyone with an interest in the differences between biden voters and trump voters in the months ahead of the 2020 election this report is largely a tab deck but also includes some of my introductory notes and a smattering of thoughts about how to approach some of the variables all the code to generate the report is available in the htad repository on github stasis a simple static site generator 30 dec 2020 czep web 3 700 words after a short tenure as the generator for this website i ve retired gatsby and replaced it with stasis a small and simple static site generator written by me in python at the risk of spending more time discussing how i generate my blog rather than actually contributing real content in this post i will describe the frustrations i encountered with gatsby the motivations i had for writing my own generator and some of the things i learned along the way i will also try to make the case that you too should write your own static site generator it s not a hard problem but it will help exercise your engineering skills in a few ways how not to tank a perfectly good power analysis 26 apr 2020 czep statistics 2 000 words power analysis is one of the most fundamental data science procedures and a prerequisite for anyone doing a b testing when reviewing the power analyses of junior data scientists i usually look first at how they estimate the variance of their target metric this is one of the trickier aspects of the process to get right and it s usually where they spend most of their time in our haste to get the variance right it s easy to overlook another even more critical piece of the estimate your target population size specifically how many unique experiment assignments you will expect over time a minimap for your data 04 apr 2020 czep data web 1 800 words sublime text has a handy feature called minimap which shows a small condensed version of your text file along the right hand margin this gives you a high level view of the file you re working in and what the file looks like if zoomed out so far that you could see all the text on one screen this can help in navigating around very large files i took some inspiration from this feature thinking that if it can be useful for text files it may also be useful for datasets i m writing an r package for exploratory data analysis in the browser with react and the minimap is the first feature that i d like to showcase to demonstrate what s possible by leveraging a front end web application to power data analysis please read a front end for eda for a more detailed introduction to this project a front end for eda 31 mar 2020 czep data web 5 100 words i am writing an r package for exploratory data analysis in the browser with reactjs in my last post i outlined my lofty ambitions for writing a graphical data analysis tool to make exploratory data analysis easier describing the motivations i had after struggling with the context switch between analysis and programming in this post i will go on a whirlwind tour of my thoughts and inspirations that will start with the front end go deep into the backend and then journey back to the front end again by the end i will hope to have made the case that i m not crazy and this thing might actually work using window functions and cross joins to count events above a threshold 24 feb 2020 czep data 4 700 words i haven t written a sql post since generating post hoc session ids in sql i don t ordinarily think of sql as good candidates for blog posts because to me sql is just boring i do use it everyday though and i ve certainly internalized a lot of handy tricks today i d like to share one of those rare moments when i sat back and thought to myself wow this query is beautiful the solution involved using not just one but two cross joins and a window function to count the number of events occurring at or above each level of a score migrating to gatsby 02 feb 2020 czep web 3 900 words i have just completed a migration of this blog from jekyll to gatsby along the way i ve learned a little bit more about react and node while taking my blog out of the flat static and bootstrap era and into the over engineered modern javascript era i still have much to learn and i still need to re style the site but at least now i have a workflow that actually works in this post i ll discuss how i decided on gatsby and a few of the specifics i encountered during the migration page 1 of 6 next subscribe via rss links home about ancient history rss feed topics data 15 linux 11 statistics 3 culture 7 web 12 work 10 recent entries this time is no different obfuscating your primary keys how to load ipums datasets into a relational database exploratory analysis with the 2020 american national election studies stasis a simple static site generator how not to tank a perfectly good power analysis a minimap for your data a front end for eda using window functions and cross joins to count events above a threshold migrating to gatsby variables and observations work data and working_with_data send feedback here