Meta tags:
description=;
author=Sung Min Park, Kristian Georgiev, Andrew Ilyas, Guillaume Leclerc, Aleksander Madry;

Headings (most frequently used words):

trak, scoring, train, top, samples, images, target, bottom, and, effective, data, attribution, is, fast, orders, of, magnitude, than, comparably, methods, attributing, model, behavior, at, scale, that, quickstart, example, scores, in, language, vision, tasks, madry, lab, installation, basic, usage, faster, more,

Text of the page (most frequently used words):
the (59), and (32), with (29), model (28), trak (26), entailment (21), prediction (20), for (18), train (18), #scoring (16), what (15), top (15), target (12), down (12), had (12), riding (12), living (11), were (11), traker (11), samples (10), are (9), have (9), under (9), caption (9), which (8), from (8), there (8), high (8), temperature (8), householder (8), present (8), images (8), batch (8), that (7), set (7), households (7), age (7), together (7), example (7), snowboard (7), man (7), motorcycle (7), blue (7), trees (7), bananas (7), laying (7), laptop (7), can (6), sports (6), out (6), record (6), children (6), couples (6), person (6), hill (6), bus (6), giraffe (6), cat (6), scores (6), computer (5), two (5), them (5), married (5), female (5), husband (5), qnli (5), white (5), field (5), use (4), does (4), bottom (4), center (4), was (4), sex (4), how (4), below (4), question (4), pairs (4), bert (4), models (4), helmet (4), next (4), table (4), data (4), more (4), fast (4), effective (4), provide (3), nanjing (3), did (3), genre (3), small (3), music (3), its (3), where (3), highest (3), 1985 (3), official (3), january (3), opposite (3), male (3), wife (3), many (3), answer (3), score (3), language (3), dataset (3), road (3), people (3), sitting (3), parked (3), sold (3), zebra (3), close (3), image (3), coco (3), clip (3), you (3), quickstart (3), code (3), attribution (3), park (2), shared (2), these (2), other (2), quantum (2), dot (2), leds (2), any (2), time (2), centers (2), wutaishan (2), olympic (2), type (2), her (2), madonna (2), not (2), greece (2), very (2), nightclub (2), junk (2), has (2), been (2), range (2), lindisfarne (2), about (2), young (2), lowest (2), records (2), kept (2), 115 (2), low (2), city (2), june (2), airport (2), 105 (2), day (2), july (2), hottest (2), fresno (2), non (2), families (2), 917 (2), 835 (2), 092 (2), 510 (2), 327 (2), percent (2), persons (2), show (2), few (2), randomly (2), selected (2), test (2), influence (2), base (2), natural (2), task (2), given (2), sentence (2), snow (2), covered (2), standing (2), bicycles (2), driving (2), large (2), some (2), bunches (2), topped (2), lots (2), ripe (2), being (2), outside (2), targets (2), latent (2), space (2), captions (2), check (2), examples (2), targets_loader (2), ckeckpoint (2), num_samples (2), shape (2), your (2), imagenet (2), train_loader (2), featurize (2), basic (2), tutorials (2), docs (2), installation (2), pip (2), install (2), version (2), cuda (2), orders (2), magnitude (2), than (2), comparably (2), methods (2), maintained, kristian, georgiev, andrew, ilyas, sung, min, only, access, previous, versions, files, stored, windows, server, subsystem, components, worked, however, still, available, software, shadow, copy, this, allows, create, almost, color, cie, diagram, special, skill, may, result, fines, sometimes, loss, one, driver, license, but, jail, usually, happens, misdemeanors, further, they, hygienic, barrier, keeping, infectious, toxic, materials, away, body, clothing, during, hazardous, activities, waste, half, carbon, fixed, calvin, cycle, rubisco, mistake, major, considered, stephen, thomas, erlewine, noted, self, titled, debut, album, began, career, disco, diva, era, such, divas, speak, started, democratic, developed, country, advanced, income, economy, quality, life, standard, political, philosophy, follow, nominated, best, plays, host, dance, acts, featured, folk, rock, group, strong, tyneside, connection, classified, though, variation, great, among, normal, chronotypes, average, human, adult, reaches, minimum, hours, before, habitual, wake, when, morning, adults, reach, university, arizona, since, 1894, maximum, tucson, recorded, within, limits, 104, 1944, february, 1899, although, historical, august, 1999, charleston, hit, coldest, extremes, ranged, most, recently, 2012, raleigh, 1905, while, 1913, 504, percentage, atlantic, made, individuals, roughly, same, 497, 2000, census, household, 158, 349, 511, 284, 547, 698, kids, predicting, inference, binary, classification, goal, predict, whether, contains, finetune, glue, benchmark, slope, ramp, backpack, rides, back, street, wearing, woman, dress, front, curb, traveling, through, behind, another, vehicle, bunch, chairs, several, each, apples, grouped, full, walks, grass, giraffes, grassy, background, near, dog, stretched, legs, open, thinking, should, containing, complex, everyday, scenes, described, free, text, learns, visual, concept, supervision, embedding, microsoft, vision, tasks, now, array, section, numpy, imagenetloader, false, start_scoring_checkpoint, num_targets, finalize_scores, exp_name, finally, get, all, validation, image_classification, train_set_size, load_checkpoint, model_id, finalize_features, then, initalize, class, process, torchvision, import, resnet18, checkpoint, state_dict, imagenetdataloader, true, first, intialize, loaders, want, minimal, pseudo, showcasing, workflow, getting, pair, depth, including, ready, run, notebooks, those, include, slurm, our, usage, details, visit, faqs, will, need, compile, require, compilation, gcc, toolkit, custom, cifar, faster, blog, post, paper, try, efficient, machine, learning, attributing, behavior, scale, home, madry, lab,

Text of the page (random words):
trak madry lab home docs tutorials trak attributing model behavior at scale effective efficient data attribution for large machine learning models try it paper blog post code data attribution that is fast and effective trak is orders of magnitude faster than comparably effective data attribution methods and orders of magnitude more effective than comparably fast methods cifar 10 qnli imagenet quickstart installation for a fast version with custom cuda code use pip install traker fast you will need cuda toolkit and gcc to compile it for the version that does not require compilation use pip install traker for more details visit the installation faqs basic usage below we provide a minimal pseudo code example showcasing the basic workflow of getting trak scores for a given model dataset pair for more in depth examples including ready to run notebooks check the tutorials in our docs those include how to use trak with bert how to set up trak with slurm and more first intialize your model and data loaders you want to score with trak for example from torchvision import models model models resnet18 checkpoint model state_dict train_loader imagenetdataloader train true then initalize the traker class and process featurize the train set traker traker model model task image_classification train_set_size traker load_checkpoint ckeckpoint model_id 0 for batch in train_loader traker featurize batch batch num_samples batch 0 shape 0 traker finalize_features finally get the trak scores for your targets e g all imagenet validation samples targets_loader imagenetloader train false traker start_scoring_checkpoint quickstart ckeckpoint num_targets for batch in targets_loader traker score batch batch num_samples batch 0 shape 0 scores traker finalize_scores exp_name quickstart that s it now you re have the trak scores in a numpy array check out the section below for some examples example trak scores in language and vision tasks clip on ms coco bert on qnli microsoft coco is a dataset containing images of complex everyday scenes described with free text captions clip is a model that learns visual concept from natural language supervision by embedding the images and captions in a shared latent space below we show the highest scoring image caption pairs for a few randomly selected targets from the ms coco test set image caption pairs from the train set with high trak scores have a high influence on clip thinking that the target image and caption should be close in its latent space target caption a cat is laying on top of a laptop computer trak top scoring train images a cat is laying on top of a laptop computer a cat laying next to an open laptop computer a dog stretched out laying on a persons legs under a laptop the cat is laying on top of the laptop a cat is laying on top of a laptop computer a cat is laying down on a white laptop target caption a close up of a giraffe and a zebra in a field near trees trak top scoring train images a giraffe standing in a field and by trees a close up of a giraffe with trees in the background two giraffes in a grassy field with small trees a giraffe walks in the field with trees and grass a giraffe and zebra together in a field the giraffe and zebra are outside by the trees target caption a table full of bananas being sold outside trak top scoring train images there are many bunches of bananas being sold bananas and apples grouped together to be sold a table topped with lots of ripe bananas a table topped with lots of ripe bananas sitting next to each other several bunches of bananas on a table a large bunch of bananas sitting by some chairs target caption a blue and white bus with two bicycles and people by it trak top scoring train images a blue and white bus parked behind another vehicle a blue and white bus driving through a park next to trees a blue bus is traveling down the road a blue bus driving down a road next to people a blue and white bus is parked by a curb a young woman in a blue dress standing in front of parked bicycles target caption a man riding a motorcycle with a helmet on trak top scoring train images a person wearing a helmet is riding a motorcycle a person in a helmet is riding a motorcycle a person with a helmet is sitting on a motorcycle two people riding a motorcycle down a street a man is riding a very small motorcycle a man riding on the back of a motorcycle down a road target caption a person riding a snowboard down a hill trak top scoring train images a person on a snowboard rides on the hill a man riding a snowboard down a snow covered hill a man riding a snowboard with a backpack down a hill a man riding a snowboard down a hill a man riding a snowboard down a hill to a ramp a person riding a snowboard down a snow covered slope qnli is a natural language inference dataset from the glue benchmark it is a binary classification task where given a question and a sentence the goal is to predict whether the sentence contains an answer to the question we finetune bert base models on qnli below we show the highest scoring question answer pairs for a few randomly selected samples from the qnli test set question answer pairs with a high trak score have a high influence on bert base predicting entailment for the target example 1 example 2 example 3 example 4 target q how many households has kids under the age of 18 living in them a there were 158 349 households of which 68 511 43 3 had children under the age of 18 living in them 69 284 43 8 were opposite sex married couples living together 30 547 19 3 had a female householder with no husband present 11 698 7 4 had a male householder with no wife present model prediction entailment trak top scoring train samples q what percent of household have children under 18 a there were 46 917 households out of which 7 835 16 7 had children under the age of 18 living in them 13 092 27 9 were opposite sex married couples living together 3 510 7 5 had a female householder with no husband present 1 327 2 8 had a male householder with no wife present model prediction entailment q what percent in the 2000 census had persons under the age of 18 a a there are 44 497 households out of which 15 8 have children under the age of 18 27 5 are married couples living together 7 5 have a female householder with no husband present and 62 3 are non families model prediction entailment trak bottom scoring train samples q roughly how many same sex couples were there a there were 46 917 households out of which 7 835 16 7 had children under the age of 18 living in them 13 092 27 9 were opposite sex married couples living together 3 510 7 5 had a female householder with no husband present 1 327 2 8 had a male householder with no wife present model prediction no entailment q what percentage of households in atlantic city were made up of individuals a there were 15 504 households of which 27 3 had children under the age of 18 living with them 25 9 were married couples living together 22 2 had a female householder with no husband present and 44 8 were non families model prediction no entailment target q what is the hottest temperature record for fresno a the official record high temperature for fresno is 115 f 46 1 c set on july 8 1905 while the official record low is 17 f 8 c set on january 6 1913 model prediction entailment trak top scoring train samples q what is the hottest temperature in raleigh a extremes in temperature have ranged from 9 f 23 c on january 21 1985 up to 105 f 41 c most recently on july 8 2012 model prediction entailment q what day did charleston s airport hit the coldest day on record a the highest temperature recorded within city limits was 104 f 40 c on june 2 1985 and june 24 1944 and the lowest was 7 f 14 c on february 14 1899 although at the airport where official records are kept the historical range is 105 f 41 c on august 1 1999 down to 6 f 14 c on january 21 1985 model prediction entailment trak bottom scoring train samples q what was tucson s record low a at the university of arizona where records have been kept since 1894 the record maximum temperature was 115 model prediction no entailment q when does the temperature of morning type young adults reach its lowest a though variation is great among normal chronotypes the average human adult s temperature reaches its minimum at about 05 00 5 a m about two hours before habitual wake time model prediction no entailment target q what genre of music is lindisfarne classified as a lindisfarne are a folk rock group with a strong tyneside connection model prediction entailment trak top scoring train samples q what genre of music is featured at junk a the nightclub junk has been nominated for the uk s best small nightclub and plays host to a range of dance music s top acts model prediction entailment q which political philosophy does greece follow a greece is a democratic and developed country with an advanced high income economy a high quality of life and a very high standard of living model prediction entailment trak bottom scoring train samples q which genre did madonna started out in a stephen thomas erlewine noted that with her self titled debut album madonna began her career as a disco diva in an era that did not have any such divas to speak of model prediction no entailment q what type of sports centers are wutaishan sports center and nanjing olympic sports center considered to be a there are two major sports centers in nanjing wutaishan sports center and nanjing olympic sports center model prediction no entailment target q what can rubisco do by mistake a it can waste up to half the carbon fixed by the calvin cycle model prediction no entailment trak top scoring train samples q what can clothing provide during hazardous activities a further they can provide a hygienic barrier keeping infectious and toxic materials away from the body model prediction no entailment q what usually happens with misdemeanors a these may result in fines and sometimes the loss of one s driver s license but no jail time model prediction no entailment trak bottom scoring train samples q quantum dot leds can do what special skill a this allows quantum dot leds to create almost any color on the cie diagram model prediction entailment q what does 76 shadow copy do a it can only access previous versions of shared files stored on a windows server computer 74 the subsystem on which these components worked however is still available for other software to use 74 model prediction entailment maintained by kristian georgiev andrew ilyas sung min park