Meta tags:
description=;

Headings (most frequently used words):

data, attribution, predictive, iv, in, the, wild, model, debugging, three, applications, wrapping, up, more, alike, than, different, setting, simple, formula, for, applying, dataset, selection, machine, unlearning, poisoning,

Text of the page (most frequently used words):
the (121), data (91), model (57), #attribution (52), training (48), for (36), this (27), and (27), that (22), methods (19), are (17), dataset (17), with (15), machine (15), hat (13), theta (13), predictive (13), unlearning (13), problem (12), from (12), sample (12), not (11), can (11), 2023 (11), then (11), outputs (11), selection (11), loss (11), 2024 (10), subset (10), adv (10), applications (10), set (10), targ (9), these (9), given (9), samples (9), example (9), each (9), left (8), right (8), models (8), function (8), step (8), which (8), poisoning (8), behavior (8), chapter (8), could (8), importance (8), wild (7), settings (7), there (7), first (7), optimization (7), target (7), over (7), trained (7), how (7), method (7), forget (7), tutorial (6), mathbb (6), sim (6), cup (6), clean (6), often (6), learning (6), part (6), where (6), 2022 (6), directly (6), optimize (6), performance (6), estimate (6), predictions (6), all (6), test (6), prediction (6), parameters (6), choice (6), why (6), potential (6), approx (5), debugging (5), still (5), large (5), many (5), one (5), use (5), maximize (5), respect (5), above (5), approximate (5), after (5), goal (5), want (5), terms (5), would (5), setting (5), best (5), common (5), ell (4), our (4), useful (4), out (4), well (4), while (4), scale (4), new (4), number (4), 2020 (4), difficult (4), based (4), adversary (4), consider (4), exactly (4), more (4), estimates (4), mapping (4), train (4), however (4), information (4), may (4), using (4), possible (4), output (4), what (4), explain (4), understand (4), mit (3), min_ (3), setminus (3), recent (3), have (3), think (3), understanding (3), improving (3), existing (3), way (3), their (3), section (3), place (3), poison (3), koh (3), process (3), plugging (3), aims (3), hurts (3), task (3), below (3), hurt (3), most (3), such (3), internet (3), like (3), benchmarks (3), much (3), without (3), generally (3), aim (3), those (3), interest (3), select (3), practice (3), straightforward (3), optimizing (3), two (3), identify (3), formula (3), three (3), framework (3), simply (3), inspecting (3), hypothesis (3), hypotheses (3), cover (3), arg (2), hope (2), found (2), any (2), years (2), known (2), tools (2), within (2), viewed (2), they (2), actually (2), continually (2), challenges (2), standard (2), discussed (2), here (2), general (2), primitive (2), applying (2), works (2), manner (2), 2018 (2), liang (2), objective (2), hard (2), through (2), had (2), performing (2), complicated (2), proxy (2), full (2), solve (2), small (2), size (2), learn (2), cifar (2), attacker (2), attack (2), modify (2), provide (2), ranging (2), under (2), across (2), both (2), modeling (2), resources (2), than (2), between (2), 2021 (2), predict (2), logits (2), approach (2), every (2), time (2), comes (2), removing (2), effect (2), remove (2), datasets (2), show (2), improve (2), datacomp (2), game (2), theoretic (2), wang (2), tend (2), simple (2), linear (2), approximation (2), downstream (2), too (2), view (2), language (2), when (2), help (2), available (2), before (2), different (2), setup (2), detail (2), separate (2), application (2), explanations (2), represent (2), identifying (2), point (2), importances (2), mean (2), briefly (2), describe (2), high (2), overview (2), see (2), analyzing (2), sunset (2), inspect (2), incorrectly (2), generating (2), site, created, jekyll, tufte, theme, icml, july, concludes, you, some, material, please, reach, comments, questions, edu, despite, rapid, progress, really, adoption, production, couple, reasons, practitioners, second, even, academia, only, descriptive, tool, studying, makes, convincing, case, building, systems, finally, accurate, enough, reliable, fundamental, particular, expect, become, toolkit, next, nguyen, forward, infancy, exciting, arising, broadly, excited, about, recipe, writes, down, plugs, versatile, wrapping, similar, though, them, explicitly, framing, fang, xiao, biggio, 2012, same, instead, replace, original, input, underset, operatorname, max, has, complete, control, fixed, knows, defender, will, combined, construct, poisoned, maximally, frog, class, depending, exact, execute, should, answer, note, defending, against, attacks, modifying, threat, concern, today, trainers, collect, sources, public, anyone, post, untrusted, third, parties, amazon, mechanical, turk, evaluating, active, development, popular, supervised, generative, include, ulira, muse, tofu, approaches, involved, outlined, due, real, world, snags, direct, connection, was, pointed, upcoming, paper, georgiev, pawelczyk, warnaco, tanno, suriyakumar, sekhari, guo, yields, therefore, just, fit, classification, points, arbitrary, estimating, effective, additional, request, delete, intractable, retrain, alternative, retraining, match, arisen, seen, particularly, relevant, deployment, scenarios, providers, must, satisfy, users, who, backed, privacy, government, regulations, personal, removed, accidentally, uploading, private, service, user, company, curation, emerging, fields, attracted, lot, curated, continue, dolma, fineweb, schoch, chhabra, jiao, xia, engstrom, solving, form, taking, cannot, circumvent, operate, stages, usually, via, order, efficient, unclear, calculate, enumerating, checking, phrased, choose, minimizes, predetermined, reduces, distribution, suite, determining, tricky, helps, seem, strange, turns, realistic, contains, spam, malformed, text, incorrect, bad, keeping, good, scrapes, greatly, vision, plug, resulting, simpler, rewrite, proceeds, steps, constitute, entirely, tasks, solvable, single, let, begin, recapping, recur, throughout, universe, algorithm, corresponding, beta, assume, access, apply, iii, now, start, describing, used, discussing, individual, separately, alike, typically, beyond, top, look, principle, components, space, other, schemes, zheng, singla, marion, konz, rosenfeld, risteski, grosse, karlaš, shah, ilyas, tang, basu, pruthi, ghorbani, zou, 2019, yeh, 2017, agnostic, patterns, distance, corroborative, fair, payoff, datapoint, divided, among, overall, utility, achieved, negative, shapley, including, means, dependent, follows, level, category, gives, obvious, mispredicted, associates, sunsets, boats, hand, indeed, mechanism, rigorously, defining, testing, image, whether, changes, scenario, corrected, photo, plane, assign, surfaces, following, important, might, range, correlating, human, interpretable, turn, determines, unfortunately, surface, mechanisms, sets, humans, meaningful, furthermore, necessarily, yield, insights, make, tractable, solution, features, unintuitive, concepts, tracing, weights, centric, made, did, crucial, observed, behave, classifier, predicts, roadmap, parts, categories, latter, final, mature, coming, four, topics,

Text of the page (random words):
r 1 chapter 2 chapter 3 chapter 4 iv data attribution in the wild in the final part of this tutorial we overview applications of predictive data attribution both mature and up and coming in large scale settings we ll cover four topics model debugging dataset selection machine unlearning data poisoning as a roadmap we cover these applications in two separate parts below first we briefly describe model debugging which is an application of all categories of data attribution methods we then cover the latter three applications data selection machine unlearning and data poisoning in a common predictive data attribution based framework model debugging we often want to understand why models behave the way they do for example we may want to understand why a classifier incorrectly predicts data attribution methods do not directly explain why a model made the predictions it did however such methods do help in a crucial step of the understanding process generating potential hypotheses that could explain the observed behavior a data centric view of model behavior methods for generating hypotheses that might explain model behavior range from tracing model weights to correlating human interpretable concepts with model outputs in this section we turn to the dataset to identify possible hypotheses after all the training dataset in large part determines trained model behavior unfortunately it is hard to surface potential mechanisms for model behavior by simply inspecting the training dataset directly training sets are generally too large for humans to understand in a meaningful way furthermore simply inspecting training data samples may not necessarily yield insights on model behavior as models learn unintuitive features from data to make analyzing the dataset more tractable one solution is data attribution debugging with data attribution consider the scenario above where we aim to understand why a cifar 10 model corrected incorrectly on a photo of a plane on a sunset a common data attribution based approach for identifying potential explanations is to assign each training sample an importance then inspect the high importance samples for example a predictive data attribution method surfaces the following most important training samples for the test sample of interest above analyzing these samples gives an obvious potential hypothesis for why the model mispredicted the model associates sunsets with boats with this hypothesis in hand we can then test to see if it is indeed a mechanism that would explain the prediction by first rigorously defining the hypothesis then actually testing it for example we could remove the sunset from the image and inspect whether it changes the prediction what exactly do these importances mean exactly what importance means is dependent on the data attribution method of choice in what follows we briefly describe at a high level what these importances could mean in each category of data attribution for a full overview of each method see chapter 1 of the tutorial predictive data attribution when the data attribution method is linear cf chapter 3 in the choice of the training data the importance for a given training sample is effect of including that sample in training on the prediction game theoretic in the data shapley setup this importance could represent the fair payoff to each training datapoint divided among the overall utility achieved by the model on the test point e g negative loss corroborative in this setting the importance could represent the distance between each training sample and the test point while this importance is model agnostic it could still be useful for identifying potential patterns in the training data in the wild there are many works on using data attribution methods to identify potential explanations for model behavior these methods typically go beyond simply inspecting the top importance samples to look at principle components in data space or other more complicated schemes koh liang 2017 yeh et al 2018 ghorbani zou 2019 pruthi et al 2020 basu et al 2020 tang et al 2021 ilyas et al 2022 shah et al 2022 karlaš et al 2022 grosse et al 2023 rosenfeld risteski 2023 konz et al 2023 marion et al 2023 singla et al 2023 zheng et al 2024 three predictive data attribution applications more alike than different we now detail three separate applications dataset selection machine unlearning and data poisoning under a common predictive data attribution framework in this section we start by describing the setting used across all the applications we then detail the common framework before discussing each individual application separately setting let s begin by recapping setup from part ii and part iii that recur throughout the applications below we consider a universe of possible training data u a learning algorithm theta mapping training subset to model parameters and a model output function f mapping an example z and model parameters theta s to a corresponding output f z beta for example f could output loss logits on the sample given a sample z of interest we assume access to a predictive data attribution estimate of the model output hat f _z s approx f z theta s which estimates the predictions of a model trained on s how can we apply such a primitive a simple formula for applying predictive data attribution while these applications constitute entirely different machine learning tasks in practice we show that each is solvable with a single straightforward formula this formula proceeds in two steps for a given task rewrite the problem as a prediction or optimization problem in terms of model outputs i e in terms of f z theta s plug in a predictive data attribution method to approximate the model outputs from the first step i e use the estimates hat f _z s approx f z theta s in place of model outputs then solve the resulting much simpler optimization problem dataset selection in dataset selection the goal is to select the best subset for model training out of all the available data this may seem like a strange goal why not train on all the training data it turns out that in many realistic settings much of the available training data hurts downstream performance for example in language and vision settings we often train on internet data which often contains spam malformed text and incorrect information in practice removing bad data and keeping good data from internet scrapes before training can greatly improve model performance however determining which data is the best is tricky this is where data attribution comes in data attribution methods help identify which training data helps and which data hurts model training step 1 dataset selection in terms of model outputs dataset selection can be phrased as an optimization problem where the goal is to choose the training data that minimizes model loss in this view we aim to select the predetermined size m subset of data s subset u that most reduces target loss loss on a given target distribution d_ targ e g in a language setting this could be a suite of common lm benchmarks when trained on or the set s arg min_ s subset u s m mathbb hat e _ z sim d_ targ left ell z theta s right however optimizing over choice of training data is difficult usually we optimize over parameters not training data choice while we can maximize with respect to parameters via first order methods there are no known efficient methods for directly optimizing over model performance with respect to the training data choice in general it is unclear how to calculate the best training subset without enumerating over every possible subset and checking for the best model step 2 plugging in data attribution we cannot optimize through model training with respect to training data the function mapping training data to target loss is too difficult to optimize directly to circumvent this problem we operate in two stages a approximate this function using data attribution then b optimize over the approximation data attribution outputs tend to be straightforward to optimize over with respect to data to approximate the function we use data attribution directly we approximate the downstream loss on each sample as taking our hat f from above as modeling the loss hat f _z s approx ell z theta s we then use this approximation to estimate the best possible training dataset selection by optimizing hat s arg min_ s subset u s m mathbb hat e _ z sim d_ targ left hat f _z s right in practice solving this optimization problem is generally straightforward as data attribution estimates tend to be simple for example a standard form is hat f _z as a linear function of the choice of training data as in chapter 3 in the wild a number of methods select datasets with data attribution ranging from those using predictive data attribution wang et al 2023 engstrom et al 2024 xia et al 2024 jiao et al 2024 chhabra et al 2024 to those using game theoretic attribution schoch et al 2023 wang et al 2024 resources data selection and curation are emerging fields that have attracted a lot of recent interest there are a number of dataset selection challenges e g datacomp and datacomp lm as well as curated datasets like fineweb and dolma that continue to show how to improve machine learning models by improving data machine unlearning machine unlearning is the task of removing the effect of training on a given forget set of training samples on model outputs this problem is particularly relevant in model deployment scenarios where providers must satisfy users who often backed by privacy first government regulations want their personal information removed from models for example after accidentally uploading private information to a service a user may want the company to remove their information from trained models an effective approach to machine unlearning would be to train a new model every time that a new forget set i e an additional request to delete a training sample comes in however in large scale model settings it is generally intractable to retrain a model for each new forget sample machine unlearning methods provide an alternative to retraining these methods aim to modify existing trained models outputs to match those that would have arisen if the models had not seen forget samples at train time step 1 machine unlearning in terms of model outputs consider existing parameters theta fit to a training dataset s and a model prediction function f mapping sample and parameters to model predictions e g logits in a classification setting given a forget set of training points f subset s the goal of unlearning is to estimate the predictions of a model trained without the forget set f for arbitrary test samples z this is exactly the problem of estimating the predictions f z theta s setminus f step 2 plugging in data attribution we can use data attribution methods to estimate these predictions after all given a test sample z our data attribution method hat f _z yields estimates of how a model would predict on z as a function of the training set therefore our unlearning prediction estimate is just the data attribution estimate of how the model would predict without training on the forget set f z theta s setminus f approx hat f _z s setminus f in the wild there are a number of methods for performing machine unlearning with data attribution based approaches and are often much more involved than the method outlined above due to real world snags guo et al 2020 sekhari et al 2021 suriyakumar et al 2022 tanno et al 2022 warnaco et al 2023 pawelczyk et al 2023 the direct connection between predictive data attribution and machine unlearning was pointed out in an upcoming paper georgiev et al 2024 resources benchmarks for evaluating machine unlearning are still under active development recent popular benchmarks across both supervised learning and generative modeling include tofu muse and ulira data poisoning in data poisoning attacks there is an attacker that aims to hurt model behavior by modifying a small part of the training dataset such a threat model is a concern in many settings as today s model trainers collect data from sources ranging from the public internet where anyone can post to untrusted third parties like amazon mechanical turk depending on the exact manner in which the attacker aims to hurt model performance this attack can be difficult to execute how exactly should one modify training data to most hurt model performance data attribution methods provide one answer note understanding how to attack is useful for defending against poisoning as well step 1 data poisoning in terms of model outputs consider a training set s s_ clean cup s_ adv for which an adversary has complete control over the small fixed size subset s_ adv the adversary knows that the defender will learn a model on the combined training dataset the goal of the adversary is to construct a set of poisoned samples s_ adv that maximally hurts the performance of the trained model theta s_ clean cup s_ adv on a target set d_ targ here d_ targ is a given task that we want to poison for example poisoning the frog cifar 10 class as below viewed as an optimization problem the adversary aims to solve maximize target loss after training underset s_ adv operatorname max mathbb e _ z sim d_ targ left ell z s_ adv cup theta s_ clean right step 2 plugging in data attribution this objective is hard to maximize directly as it is difficult to optimize through the model training process with respect to input data this is the same problem we had performing dataset selection above model training is a complicated function instead we can replace the model training process with a data attribution based proxy one can approximate the target loss after training on a given training dataset as mathbb e _ z sim d_ targ left hat f _z left s_ adv cup s_ clean right right approx mathbb e _ z sim d_ targ left ell z theta s_ clean cup s_ adv right then maximize this proxy in place of the original objective the full optimization problem is then to maximize loss on the target min_ s_ adv mathbb e _ z sim d_ targ left hat f _z left s_ adv cup s_ clean right right in the wild there are a number of works poison that poison training data in a similar manner though many of them do not explicitly use the data attribution framing biggio et al 2012 koh liang 17 xiao et al 2018 fang et al 2020 koh et al 2022 wu et al 2023 wrapping up many of the applications that we ve discussed here are still in their infancy and exciting new applications of data attribution are continually arising broadly we are excited about the general recipe discussed in this section where one first writes down an optimization problem then plugs in data attribution in place of model outputs as a versatile primitive for applying predictive data attribution the way forward for data attribution despite rapid progress in recent years data attribution methods have not really found adoption in production settings we think there are a coupl...