Meta tags:
description=The proper study of mankind is manifolds;

Headings (most frequently used words):

2010, november, of, december, 17, svd, the, for, remember, big, numbers, going, to, be, out, commission, january, 11, 2011, all, done, nasa, finds, new, life, multiscale, planes, and, geometric, wavelets, 30, vision, without, categories, custom, essay, man, 16, shape, google, 15, alternating, methods, fifth, advice, calculus, students, archives, recent, posts, blogroll, feeds,

Text of the page (most frequently used words):
the (180), and (72), for (45), this (44), that (37), you (29), are (26), have (23), with (21), here (20), 2010 (18), can (18), there (17), matrix (16), all (14), #manifold (14), data (13), what (12), from (12), sarah (12), svd (11), these (11), but (11), noise (11), like (10), image (10), don (10), some (10), also (10), posted (10), uncategorized (10), really (10), very (10), different (10), dimensional (9), life (9), wavelets (9), vision (9), november (9), not (9), how (9), than (9), was (9), then (9), where (9), methods (8), high (8), planes (8), going (8), problem (8), more (8), singular (8), small (8), point (8), two (8), dimension (8), get (7), multiscale (7), geometric (7), out (7), your (7), school (7), tags (7), well (7), write (7), rank (7), kernel (7), heat (7), while (7), points (7), curvature (7), vector (6), random (6), learning (6), based (6), computer (6), done (6), look (6), know (6), they (6), set (6), which (6), each (6), large (6), values (6), comment (6), looking (6), other (6), feature (6), one (6), big (5), numbers (5), comments (5), new (5), analysis (5), pca (5), maggioni (5), geometry (5), dimensionality (5), draw (5), will (5), level (5), math (5), alternating (5), form (5), given (5), matrices (5), columns (5), value (5), any (5), etc (5), about (5), only (5), bag (5), features (5), objects (5), just (5), paper (5), plane (5), content (4), now (4), blog (4), theory (4), machine (4), operator (4), categories (4), search (4), time (4), help (4), because (4), actually (4), generally (4), students (4), work (4), database (4), faces (4), first (4), eigenvalues (4), own (4), use (4), gaussian (4), text (4), words (4), should (4), their (4), decay (4), norm (4), approximation (4), solution (4), interesting (4), databases (4), them (4), into (4), space (4), laplacian (4), would (4), way (4), doing (4), has (4), think (4), something (4), add (4), tangent (4), many (4), radius (4), ball (4), wordpress (3), com (3), sign (3), zero (3), mauro (3), processing (3), grad (3), reduction (3), retrieval (3), algebraic (3), academia (3), without (3), nasa (3), finds (3), posts (3), may (3), december (3), pictures (3), picture (3), too (3), most (3), example (3), taking (3), long (3), who (3), right (3), calculus (3), lot (3), advice (3), reading (3), good (3), kind (3), practice (3), same (3), step (3), between (3), least (3), low (3), smaller (3), documents (3), used (3), sparse (3), instance (3), range (3), real (3), want (3), completely (3), shape (3), shapes (3), dna (3), description (3), its (3), computing (3), representing (3), rather (3), images (3), sets (3), scales (3), when (3), idea (3), people (3), person (3), study (3), graduate (3), him (3), papers (3), tomasz (3), principal (3), dictionary (3), face (3), dimensions (3), locally (3), estimating (3), manifolds (3), site (2), log (2), subscribed (2), subscribe (2), already (2), account (2), top (2), full (2), spherical (2), harmonics (2), topology (2), teaching (2), structure (2), motion (2), formula (2), mean (2), laplace (2), beltrami (2), isomap (2), transform (2), stuff (2), july (2), arthur (2), szlam (2), commission (2), recent (2), january (2), 2011 (2), calc (2), result (2), cool (2), save (2), could (2), relevant (2), seems (2), hard (2), say (2), must (2), isn (2), always (2), through (2), computations (2), worry (2), currently (2), highly (2), recommend (2), writing (2), got (2), day (2), radical (2), remember (2), decaying (2), fast (2), slowly (2), works (2), coming (2), our (2), calculate (2), decomposition (2), orthogonal (2), squares (2), problems (2), solved (2), explicitly (2), using (2), entries (2), powers (2), hit (2), estimate (2), document (2), word (2), often (2), expected (2), taken (2), over (2), terms (2), likely (2), however (2), known (2), why (2), fix (2), let (2), find (2), best (2), explicit (2), eigenvectors (2), means (2), both (2), last (2), applied (2), notes (2), far (2), useful (2), google (2), tested (2), various (2), transformations (2), changes (2), scale (2), hamming (2), modern (2), represents (2), quantization (2), authors (2), since (2), discrete (2), case (2), instead (2), seen (2), function (2), surface (2), signature (2), depends (2), local (2), information (2), formats (2), extra (2), much (2), represented (2), clouds (2), produce (2), compared (2), distance (2), looks (2), make (2), article (2), chance (2), bad (2), whole (2), student (2), wouldn (2), thought (2), start (2), see (2), pay (2), apparently (2), propsal (2), need (2), place (2), somewhat (2), man (2), custom (2), essay (2), obviously (2), sensible (2), today (2), come (2), except (2), algorithm (2), called (2), cube (2), construct (2), such (2), estimated (2), supported (2), situation (2), signal (2), fact (2), assumed (2), enough (2), variance (2), intrinsic (2), number (2), particular (2), falls (2), covariance (2), tube (2), picks (2), still (2), within (2), curved (2), direction (2), molecules (2), phosphorus (2), holiday (2), introductory (2), proper (2), mankind (2), started, design, collapse, bar, manage, subscriptions, view, reader, report, privacy, create, free, website, feeds, tombone, geomblog, quomodocumque, nuit, blanche, jimmy, concrete, nonsense, chapter, brooke, blogroll, yves, meyer, william, vollman, swarms, speculation, spectral, selberg, trace, schur, lemma, richard, baraniuk, representation, princeton, pierre, louis, lions, persistent, homology, pcmi, partha, niyogi, noam, sobel, neuroscience, nadakuditi, field, magnetostatics, joyce, jared, tanner, reconstruction, hyperspectral, imaging, hilbert, harmonic, gunnar, carlsson, girl, fourth, diffusion, cryo, conformal, maps, science, compressed, sensing, bloomsday, anna, gilbert, andrew, algebra, april, june, august, september, october, archives, older, mistakes, multivariable, drawing, never, hopital, rule, friend, seriously, notation, bacon, magical, superpowers, ask, worksheet, documentation, xkcd, tech, support, cheat, sheet, mathematica, freak, unfairly, mistake, being, taught, professors, gauge, difficulty, tend, underestimate, takes, finish, homework, harder, false, steps, holding, office, hours, classes, enjoy, impressed, general, ought, hold, intro, class, charles, siegel, series, clearest, loveliest, ever, interested, topic, beginning, guy, fawkes, ideals, fifth, turns, labeled, next, back, upper, triangular, iterate, alternate, updates, related, completion, subset, fill, rest, simplest, approach, memory, intensive, intermediate, method, comes, drive, down, times, depending, power, happens, unfortunately, term, collection, rows, count, entry, indicating, little, imagination, indicate, importance, kinds, semantic, 1th, error, therefore, bound, danger, super, guarantee, quality, pretend, unlikely, whatever, left, probably, preserve, does, oversampling, parameter, wasteful, attempts, faster, diagonal, costs, operations, take, hat, sense, frobenius, week, seminar, talking, putting, several, results, presented, proof, project, suspect, ability, prove, list, current, engines, bing, opposed, metadata, shapegoogle, against, possible, isometries, holes, performed, outperforming, part, clock, matching, compare, embedding, distances, happen, quick, compute, architectures, cpu, giving, distribution, nearby, after, generalize, notion, spatiall, sensitive, exponentially, truncate, sum, computation, discretization, process, signatures, recall, fundamental, equation, transition, density, brownian, give, invariant, under, isometric, deformations, riemannian, metric, captures, neighborhood, global, proven, continuous, map, preserving, isometry, challenges, conformations, body, poses, complicated, usually, affine, typically, pixels, meshes, portable, universal, system, coordinates, aims, performing, comparing, bags, plain, old, euclidean, main, parts, task, detection, employed, define, dense, descriptor, selects, descriptors, possibilities, sift, maxima, persist, mser, show, smallest, variation, area, traversing, graph, searchable, engine, responds, queries, alex, bronstein, makes, lots, degrees, sake, escape, job, market, pursuing, passion, talent, realistic, paying, off, disturbing, economy, briefs, law, successful, lawyer, phd, thesis, pejoratively, inferior, gifted, academics, myself, mathematician, future, uncertain, unrealistic, goal, were, overwhelmed, tempted, tried, tutors, adjusting, sleep, schedule, reevaluating, plans, insane, decision, justified, run, someone, dante, customers, goes, pseudonym, professional, terrible, writers, asking, did, business, ethics, approved, pls, wonder, disturbed, read, chronicle, higher, education, written, writes, company, sad, common, academic, dishonesty, shocked, else, says, hey, wait, minute, qual, pass, month, throws, her, hands, dismay, usual, panoply, links, reimagining, thinks, closer, actual, considered, necessary, his, ideas, informed, wittgenstein, vannevar, bush, contemporary, research, sounds, stirring, intimidating, researchers, renaissance, men, beyond, received, publication, worth, memex, mention, across, hooked, malisiewicz, peter, jones, construction, beta, summary, efficient, storage, dan, lemire, wavelet, cloud, compressing, analyzing, online, forth, shorter, descriptive, partition, dyadic, tree, components, coarsest, divide, subsets, center, grid, approximates, keep, track, differences, smoothness, corrections, quickly, finitely, approximator, tradeoff, size, sparsity, identity, silly, compress, follows, vectors, worked, misclassified, 640, remarkable, yale, had, photos, angles, lighting, conditions, been, recognize, viewed, frontal, profile, heterogeneity, denoise, cluster, updated, estimates, almost, composed, zeros, ones, either, quite, binary, close, affinity, coordinate, nearest, neighbors, collaborators, developed, sometimes, possibly, recognition, measures, varies, regions, others, measure, reason, principle, shouldn, vary, probability, determine, outperforms, apart, presence, tell, sphere, assumption, grows, differently, respect, normal, computed, true, alternative, around, hollow, concentrated, itself, less, larger, begins, pick, balls, sweet, spot, interval, separated, haven, caught, yet, accurately, blogged, earlier, previous, volume, confined, grow, chosen, possibility, patched, together, somehow, linear, hyperplane, patches, dilemma, confuse, second, highest, terminology, plus, white, ambient, deal, lie, examples, collections, configurations, priori, ned, mechanisms, went, talk, presentation, official, announcement, live, video, scientist, felisa, wolfe, simon, discovered, molecular, earth, made, carbon, hydrogen, oxygen, nitrogen, sulfur, single, celled, organisms, arsenic, alien, creature, microbe, stop, check, those, non, corny, movie, seeing, hulu, cary, grant, katherine, hepburn, surprisingly, story, following, path, finished, finals, knows, relief, moment, finally, getting, room, clean, laundry, presents, bought, social, reanimated, textbook, focus, hiatus, few, months, again, passed, hurdles, jump, navigation, home,

Text of the page (random words):
cipal value representing the tangent direction with the second highest principal value representing the direction of curvature maggioni s alternative is multiscale svd we fix a point the noise around the manifold looks like a hollow tube most of the noise is concentrated at a radius of from the surface of the manifold itself so looking at a small ball smaller than the radius of the tube picks up less noise looking at a large ball picks up all the noise and looking at a still larger ball begins to pick up the curvature of the manifold we do svd within all these balls and look at the singular values over a range of scales there s a sweet spot a particular interval of scales where the signal singular values have separated from the noise singular values but the curvature singular values haven t caught up yet this accurately represents the curvature of the manifold i blogged about this earlier here and the paper is here the assumption about the data is that the tangent covariance grows differently with respect to the radius than the normal covariance this can be computed to be true explicitly for manifolds of co dimension one for instance the result is that if is assumed to have small enough curvature and is assumed to have small enough variance then with high probability we can determine the intrinsic dimension with only points compared to a large number of other dimension estimating methods this outperforms all of them isomap in particular really falls apart in the presence of noise and can t tell a 6 dimensional sphere from a 6 dimensional cube one interesting fact is that many real life data sets have dimensionality that varies a database of text documents for instance is low dimensional in some regions and very very high dimensional in others we measure dimensionality locally so there s no reason in principle that it shouldn t vary measures supported on k planes sometimes high dimensional data is supported on a set of planes possibly of different dimensions we don t know how many planes there are which planes they are or what their dimensions are this is the situation in face recognition it s also relevant to image and signal processing more generally dictionary learning sparse approximation etc maggioni and collaborators developed an algorithm for doing this 1 draw random points and nearest neighbors do multiscale svd at these points produce an estimated plane and its estimated dimension also estimate the noise level 2 construct an affinity matrix with the i j th coordinate this is almost a matrix composed of zeros and ones either a point is on a plane in which we get one or a point is not on that plane in which case we get 0 because the distance is large in practice with noise and curvature it s not quite binary but it s close 3 denoise and cluster this matrix and find updated estimates for the planes from this tested on a face database this worked very well it misclassified only 3 faces out of 640 what s remarkable here is that the yale face database used had 64 different photos of each person taken at different angles different lighting conditions etc this has always been a hard problem for computer vision how do you recognize that two pictures are the same person viewed frontal and profile while two other pictures are of two different people apparently this is a solution to such heterogeneity problems geometric wavelets the problem of dictionary learning is as follows given points in and a data matrix construct a dictionary of m vectors such that where is a sparse vector there s a tradeoff here between the size of and the sparsity of obviously you could just let be the whole of and then would just be the identity but that would be silly it wouldn t compress the data at all one way of doing this is something called geometric wavelets paper here shorter more descriptive paper with cool picture here given a manifold we partition it into a dyadic tree we do principal components analysis on the coarsest scale and get a plane then we divide that into two smaller subsets do pca on each of them and get two new planes for each plane at the center point of that cube in the manifold grid we have a tangent vector which approximates the tangent to the manifold the wavelets keep track of the differences if we have some smoothness in the manifold these corrections decay quickly so finitely many wavelets are a good approximator for any point there s a fast algorithm for computing this very like the wavelet transform except for a point cloud rather than a function this is useful for compressing databases analyzing online data and so forth the idea is very like peter jones construction of beta numbers summary here except that it may be more efficient in terms of storage more on geometric wavelets from dan lemire vision without categories november 17 2010 posted by sarah in uncategorized tags ai computer vision machine learning add a comment i d just like to mention that i ve come across tomasz malisiewicz s blog on machine learning and computer vision and i m hooked you should be too there s the usual panoply of links to interesting papers but then there s also tomasz s radical idea for reimagining computer vision using a memex instead of a set of categories he thinks that the vision problem will be solved by something much closer to actual ai than is generally considered necessary today his ideas are informed by wittgenstein and vannevar bush as well as contemporary research it sounds interesting to say the least then there s also tomasz s stirring if somewhat intimidating advice to students and researchers to be renaissance men and look beyond the a and the well received publication all in all very worth reading sensible sarah says hey wait a minute i thought i was a math student what s up with all this vision stuff and i have a qual to pass in a month sensible sarah throws up her hands in dismay the custom essay man november 16 2010 posted by sarah in uncategorized tags academia 3 comments i was somewhat disturbed to read this article in the chronicle of higher education it s written by a man who writes papers for a custom essay company students pay him to write their papers for them obviously it s sad to see how common academic dishonesty is but what really shocked me was something else a lot of ed dante s customers he goes by a pseudonym are in graduate or professional school and apparently they re really terrible writers when they write him asking him for help it s something like you did me business ethics propsal for me i need propsal got approved pls can you will write me paper i have to wonder why are these people in graduate school in the first place i don t mean that pejoratively i don t think you re inferior as a person if you re bad at writing or not gifted at academics but look i m a grad student myself i m here because i want to be a mathematician while the future is uncertain for all of us i wouldn t be here if i thought that was a completely unrealistic goal if i were so overwhelmed by the work here that i was tempted to pay someone to do it for me and if i d already tried tutors adjusting my sleep and study schedule etc then i d start reevaluating my plans it would be insane not to the way i see it going to graduate school is a decision that has to be justified you should do it only if you think it ll help you in the long run if you can t write your own briefs in law school you really think you re going to be a successful lawyer if you can t write your own phd thesis do you really think you have any chance to make it in academia this article really makes me worry that lots of people are going for extra degrees just for the sake of doing it or to escape the job market and not because they re really pursuing a passion or talent that has a realistic chance of paying off that s disturbing it seems like a bad sign for the economy as a whole shape google november 15 2010 posted by sarah in uncategorized tags content based image retrieval image processing laplace beltrami operator machine learning 2 comments currently reading this paper by alex bronstein et al the idea here is to make searchable databases for shapes 2d and 3d objects in the same way a text engine responds to search queries there are two main parts to this task feature detection and feature description different methods can be employed to define what a feature is a dense descriptor just selects all the points in the image as descriptors but there are other possibilities sift looks for local maxima of the discrete image laplacian that persist through different scales mser finds level sets that show the smallest variation of area when traversing the level set graph feature description aims to produce a bag of words from the features by performing vector quantization on the feature space two images can then be compared by comparing their bags of features with plain old euclidean distance there are extra challenges if we want to do this with 3d objects rather than images for one conformations of 3d objects like a body in different poses are much more complicated than the usually affine transformations seen on a given image also while images are typically represented as matrices of pixels 3d objects can be represented as meshes point clouds level sets etc so computations have to be portable between formats also 3d objects don t have a universal system of coordinates the feature description process used by the authors is based on heat kernel signatures recall that the heat kernel is the fundamental solution of the heat equation the heat kernel can also be seen as the transition density function of a brownian motion for any point on the surface we give its heat kernel signature as this is invariant under isometric deformations of the space since it depends only on the riemannian metric it captures local information in a small neighborhood of x for small t and global information for large t it can be proven that any continuous map preserving the heat kernel signature must be an isometry and computing it depends on computing the eigenvalues of the laplacian which can be done with various formats of representing 3d shapes so since the eigenvalues of the laplacian decay exponentially we can truncate this sum for computation of the heat kernel in the discrete case instead of the laplacian we would use a discretization of the form after vector quantization to get a bag of features the authors generalize this notion to a spatiall sensitive bag of features giving a matrix that represents the distribution of nearby words from there we can compare these matrices by embedding them into a hamming space hamming distances happen to be very quick to compute on modern cpu architectures shapegoogle was tested against various possible transformations of shapes isometries topology changes holes noise scale changes and performed well outperforming other methods like shape dna part based bag of words and clock matching bag of features it s an interesting project and i suspect that the ability to search shape databases will prove useful here s a list of current content based image retrieval search engines both google and bing use these methods as opposed to only looking at metadata alternating methods for svd november 9 2010 posted by sarah in uncategorized tags dimensionality reduction pca random matrix theory 1 comment so far last week s applied math seminar was arthur szlam talking about alternating methods for svd i m putting up some notes here several results are presented without proof given an m x n matrix a we want to find the best rank k approximation to a there is an explicit solution to this problem known as the singular value decomposition we write where is diagonal are orthogonal the columns of are the eigenvectors of and the columns of are the eigenvectors of this costs operations and it s a completely explicit solution to get a rank k approximation we take where the hat means taking the first k columns of each matrix this is the best rank k approximation both in the sense of frobenius norm and operator norm however if time is wasteful there are recent attempts to do this faster in time we fix k and a small oversampling parameter p and let draw an random gaussian matrix and set calculate the svd and set to be the first k columns of etc why does this work well pretend a is really of rank 5 for instance hit a with a random matrix it is unlikely that any vector will be in the kernel of a whatever s left is in the range so we re very likely to have the full range of a we probably preserve the svd however in real life we don t have a known rank k matrix what we have is a guarantee of quality in terms of the decay of the singular values here the expected value is taken over the random gaussian draw the norm is the operator norm and is the k 1th singular value if a is really rank k this is zero and the expected error is therefore zero if is small then the bound is good we re in danger if m is large and is not super small this actually happens unfortunately for example in term document matrices where a collection of text documents form the rows and words form the columns with a word count in each matrix entry indicating how often a word was used in each document a little imagination should indicate the importance of these kinds of matrices for semantic analysis these matrices are sparse and large but their singular values decay slowly so what can we do here we hit by some more powers of a to drive down the smaller eigenvalues in practice 10 or 15 times now we have a new estimate depending on the power q this works but it can be memory intensive to save all the intermediate powers this is where the alternating method comes in here we look at a related problem matrix completion you re given a subset of the entries of a matrix a and you know a is low rank how do you fill in the rest of the entries the simplest approach is alternating least squares write where p is m by k and q is n by k alternate between updates of p and q these are least squares problems and can be solved explicitly using step 1 form gaussian step 2 form matrix st a qr decomposition with q orthogonal and r upper triangular step 3 iterate for coming back to our own problem we can use the same formula to calculate this turns out to work well in practice the example set was a database of labeled faces which have the first 20 eigenvalues decaying fast then the next decaying very slowly this kind of alternating svd works well for faces remember remember the fifth of november november 5 2010 posted by sarah in uncategorized tags algebraic geometry 2 comments guy fawkes day it s a good day for radical ideals no not that kind this kind i m currently reading charles siegel s series of posts on algebraic geometry from the beginning and i highly recommend it s some of the clearest loveliest writing ever and it s got me really interested in the topic advice for calculus students november 3 2010 posted by sarah in unc...