Meta tags:
Headings (most frequently used words):
learning, self, supervised, contrastive, contents, pseudo, labels, types, comparison, with, other, forms, of, machine, examples, references, further, reading, external, links, autoassociative, non, joint, embedding, and, predictive, architectures,
Text of the page (most frequently used words):
learning (119), the (93), #supervised (57), self (56), and (43), data (32), neural (27), that (25), for (25), model (21), #machine (20), from (18), network (18), doi (18), arxiv (17), with (16), training (16), this (15), edit (15), ieee (14), conference (14), contrastive (14), input (14), are (13), iccv (13), representation (12), used (12), vision (11), deep (11), task (11), computer (10), models (10), s2cid (10), international (10), 2019 (10), using (9), lecun (9), language (9), image (9), artificial (9), latent (9), pseudo (9), examples (9), labels (9), wikipedia (8), text (8), visual (8), yann (8), intelligence (8), embedding (8), proceedings (8), unsupervised (8), 978 (8), isbn (8), 1109 (8), joint (8), can (8), networks (8), example (8), other (8), positive (8), retrieved (7), based (7), systems (7), recognition (7), context (7), images (7), processing (7), predictive (7), semi (7), learn (7), where (7), may (6), non (6), autoencoder (6), architectures (6), speech (6), human (6), loss (6), 2015 (6), 2017 (6), 2021 (6), analysis (6), which (6), not (6), negative (6), classification (6), displaystyle (6), left (6), right (6), search (5), page (5), generative (5), architecture (5), andrew (5), reasoning (5), automated (5), approach (5), gradient (5), reinforcement (5), method (5), regression (5), 2018 (5), pmid (5), issn (5), doersch (5), carl (5), october (5), 2023 (5), 2020 (5), june (5), representations (5), jepa (5), noise (5), estimation (5), random (5), autoassociative (5), machines (5), its (5), ncssl (5), ssl (5), sample (5), pairs (5), contents (4), policy (4), use (4), articles (4), technical (4), july (4), convolutional (4), state (4), term (4), john (4), knowledge (4), large (4), world (4), diffusion (4), autonomous (4), word (4), engineering (4), association (4), computational (4), 2022 (4), wang (4), april (4), micron (4), prediction (4), links (4), bardes (4), adrien (4), own (4), new (4), main (4), shot (4), transfer (4), path (4), statistical (4), facebook (4), concept (4), uses (4), labeled (4), encoder (4), create (4), process (4), output (4), such (4), between (4), types (4), log (4), vector (4), hide (4), move (4), sidebar (4), toggle (3), view (3), about (3), privacy (3), september (3), january (3), too (3), 2025 (3), short (3), wikidata (3), art (3), video (3), recurrent (3), transformer (3), alex (3), fei (3), semantic (3), list (3), audio (3), general (3), agent (3), algorithm (3), symbolic (3), open (3), datasets (3), descent (3), clustering (3), variance (3), history (3), linguistics (3), november (3), methods (3), robust (3), 4673 (3), multi (3), quentin (3), program (3), repair (3), diagnostics (3), 1038 (3), jean (3), emnlp (3), natural (3), supervision (3), few (3), via (3), 2016 (3), cvpr (3), pattern (3), features (3), multimodal (3), reduction (3), 2013 (3), correlation (3), detection (3), drift (3), through (3), humans (3), unlabeled (3), trained (3), instead (3), small (3), learns (3), two (3), only (3), way (3), itself (3), however (3), pretext (3), unlike (3), one (3), requiring (3), does (3), these (3), supervisory (3), signals (3), then (3), autoencoders (3), space (3), rather (3), than (3), has (3), while (3), without (3), would (3), samples (3), function (3), pair (3), when (3), help (3), related (3), field (3), article (3), tools (3), languages (2), table (2), statistics (2), safety (2), contact (2), under (2), terms (2), was (2), last (2), 2026 (2), hidden (2), categories (2), 2024 (2), description (2), different (2), category (2), impact (2), applications (2), principle (2), act (2), gan (2), mamba (2), rnn (2), perceptron (2), gru (2), long (2), lstm (2), turing (2), daniel (2), jan (2), david (2), oriol (2), vinyals (2), yoshua (2), bengio (2), geoffrey (2), hinton (2), oliver (2), rule (2), openai (2), ibm (2), gpt (2), synthesis (2), alexnet (2), protocol (2), neuro (2), source (2), coding (2), improvement (2), rlhf (2), sarsa (2), augmentation (2), regularization (2), bias (2), tradeoff (2), software (2), projects (2), algorithms (2), glossary (2), yarowsky (2), sense (2), zheng (2), xin (2), yong (2), guoyou (2), liu (2), jianguo (2), 3796689 (2), 29425969 (2), 0968 (2), 4328 (2), 1016 (2), 010 (2), 107 (2), fast (2), segmentation (2), white (2), blood (2), cell (2), gupta (2), abhinav (2), efros (2), alexei (2), december (2), 1430 (2), 9062671 (2), 8391 (2), 167 (2), 1505 (2), 05192 (2), 1422 (2), zisserman (2), 2079 (2), 473729 (2), 5386 (2), 1032 (2), 226 (2), 1708 (2), 07860 (2), 2070 (2), external (2), mark (2), florian (2), garrido (2), pierre (2), further (2), reading (2), martin (2), 1145 (2), acm (2), selfapr (2), test (2), execution (2), pmc (2), efficient (2), genomics (2), bootstrap (2), your (2), 2010 (2), empirical (2), google (2), bert (2), pre (2), vincent (2), abstract (2), aaai (2), ponce (2), motion (2), content (2), assran (2), mahmoud (2), ballas (2), nicolas (2), misra (2), ishan (2), michael (2), towards (2), nature (2), variable (2), modelling (2), vicreg (2), invariance (2), covariance (2), pmlr (2), barlow (2), twins (2), canonical (2), jmlr (2), workshop (2), 1476 (2), 355 (2), bibcode (2), com (2), pdf (2), system (2), computing (2), adaptation (2), icml (2), label (2), wav2vec (2), perez (2), 7281 (2), 4803 (2), cvf (2), boosting (2), what (2), like (2), references (2), across (2), diverse (2), fields (2), leverage (2), application (2), domains (2), applied (2), version (2), predictor (2), being (2), given (2), produced (2), results (2), understand (2), particularly (2), each (2), most (2), important (2), information (2), reconstruction (2), current (2), often (2), tasks (2), involves (2), designed (2), require (2), inherent (2), structures (2), generate (2), same (2), time (2), extracted (2), comparison (2), forms (2), jepas (2), structure (2), decision (2), aims (2), they (2), class (2), maximizing (2), agreement (2), constraints (2), cca (2), enforce (2), local (2), zero (2), binary (2), classify (2), requires (2), online (2), side (2), target (2), encoding (2), dimensional (2), into (2), classifier (2), birds (2), distance (2), reconstruct (2), closely (2), during (2), typically (2), difference (2), original (2), error (2), meaningful (2), mean (2), essential (2), predictions (2), risk (2), components (2), manual (2), labeling (2), instance (2), predicted (2), ground (2), truth (2), make (2), recent (2), paradigm (2), relationships (2), theory (2), curve (2), net (2), forest (2), factor (2), anomaly (2), bayes (2), structured (2), dimensionality (2), feature (2), appearance (2), upload (2), file (2), changes (2), read (2), account (2), donate (2), menu (2), add, topic, mobile, cookie, statement, developers, code, conduct, legal, contacts, disclaimers, available, additional, apply, site, you, agree, registered, trademark, profit, organization, wikimedia, foundation, inc, creative, commons, attribution, sharealike, license, rendered, parsoid, edited, utc, dmy, dates, all, https, org, index, php, title, supervised_learning, oldid, 1375949950, workplace, warfare, military, games, marketing, chatbot, psychosis, healthcare, fiction, education, engine, optimization, explainable, environmental, competition, arms, race, anthropomorphism, winter, slop, literacy, infrastructure, effect, center, bubble, boom, social, economic, opposition, centers, propaganda, virtual, politician, regulation, precautionary, nationalism, ethics, elections, takeover, alignment, government, cold, war, political, graph, gnn, adversarial, variational, vae, highway, residual, cnn, multilayer, mlp, echo, gated, unit, memory, vit, differentiable, françois, chollet, kokotajlo, leike, mustafa, suleyman, schulman, aidan, gomez, noam, shazeer, ashish, vaswani, andrej, karpathy, silver, demis, hassabis, ian, goodfellow, quoc, ilya, sutskever, krizhevsky, james, goodnight, graves, stephen, grossberg, lotfi, zadeh, jürgen, schmidhuber, hopfield, paul, werbos, seppo, linnainmaa, seymour, papert, joseph, weizenbaum, bernard, widrow, frank, rosenblatt, selfridge, herbert, simon, cliff, shaw, allen, newell, nathaniel, rochester, mccarthy, marvin, minsky, takeo, kanade, kunihiko, fukushima, shun, ichi, amari, claude, shannon, christopher, manning, von, neumann, walter, pitts, warren, sturgis, mcculloch, alan, people, yago, dbpedia, conceptnet, bases, opencog, lida, clarion, soar, cognitive, reasoners, procedural, logic, programs, inference, engines, expert, deductive, classifiers, robot, control, autogpt, action, selection, muzero, driving, car, five, alphazero, alphago, decisional, watsonx, watson, project, debater, oasis, genie, udio, suno, riffusion, music, generation, veo, seedance, sora, kling, hailuo, runway, gen, dream, stable, recraft, midjourney, imagen, ideogram, flux, firefly, dall, aurora, alphafold, facial, whisper, elevenlabs, ocr, hwr, wavenet, implementations, physical, agent2agent, hypothetical, superintelligence, asi, agi, weak, lethal, weapons, laws, humanity, exam, companion, intelligent, nmt, game, playing, theorem, proving, actor, critic, situated, sovereign, blended, vibe, hallucination, recursive, reflection, llm, post, uncanny, valley, rag, adversary, autoregression, imitation, prompt, weight, initialization, gating, rectifier, sigmoid, softmax, activation, batchnorm, normalization, convolution, attention, backpropagation, conjugate, quasi, newton, sgd, overfitting, double, functions, hyperparameter, parameter, constraint, satisfaction, planning, concepts, lists, proprietary, institutions, companies, timeline, 1995, cambridge, 196, 3115, 981658, 981684, 189, 33rd, annual, meeting, disambiguation, rivaling, balestriero, randall, ibrahim, sobal, vlad, morcos, ari, shekhar, shashank, goldstein, tom, bordes, mialon, gregoire, tian, yuandong, schwarzschild, avi, wilson, gordon, geiping, jonas, fernandez, cookbook, 2304, 12210, martinez, matias, luo, xiapu, zhang, tao, monperrus, 4503, 9475, 3551349, 3556926, 37th, gündüz, hüseyin, anil, binder, xiao, yin, mreches, rené, bischl, bernd, mchardy, alice, münch, philipp, rezaei, mina, 928, 37696966, 10495322, 2399, 3642, s42003, 023, 05310, communications, biology, grill, bastien, strub, altché, florent, tallec, corentin, richemond, buchatskaya, elena, pires, bernardo, avila, guo, zhaohan, azar, mohammad, gheshlaghi, piot, bilal, 2006, 07733, wilcox, ethan, qian, peng, futrell, richard, kohita, ryosuke, levy, roger, ballesteros, miguel, stroudsburg, usa, 4652, 222291675, 18653, 375, 05725, 4640, structural, improves, syntactic, generalization, blog, sourcing, francois, lavet, precup, doina, pineau, joelle, combined, 1809, 04506, littwin, etai, wolf, lior, 3966, 6517610, 8851, 429, 1511, 09033, 3957, multiverse, 2307, 12698, zhengcong, fan, mingyuan, huang, junshi, listen, 2311, 15830, najman, laurent, leveraging, 2403, 00504, duval, bojanowski, piotr, pascal, rabbat, 2301, 08243, openreview, ing, andrades, alvaro, cosenza, marco, raffaele, korbel, 1075, 40709098, 12283373, 2522, 5839, s42256, 025, 01052, 1053, integrating, cancer, 2105, 04906v3, zbontar, jure, jing, deny, stephane, 12320, 12310, 38th, redundancy, galen, arora, raman, bilmes, jeff, livescu, karen, 1255, 1247, 30th, gutmann, hyvärinen, aapo, march, 304, 297, thirteenth, unnormalized, oord, aaron, van, den, yazhe, 1807, 03748, becker, suzanna, 1992, 6356, 163, 1729650, 4687, 355161a0, 1992natur, 161b, 161, organizing, discovers, surfaces, dot, stereograms, demystifying, key, technique, kramer, 1991, 243, 1002, aic, 690370209, 1991aiche, 233k, 233, aiche, journal, nonlinear, principal, component, deng, chen, european, symposium, security, euros, eurosp, 00014, evolving, android, malware, gama, joão, 2014, 2523813, surveys, survey, lee, dong, hyun, challenges, wrepl, simple, gidaris, spyros, bursuc, andrei, komodakis, nikos, patrick, cord, matthieu, 8067, 186206588, 00815, 1906, 05186, 8058, beyer, lucas, zhai, xiaohua, avital, kolesnikov, alexander, 1485, 167209887, 00156, 1905, 03670, s4l, bouchard, louis, medium, will, ever, able, continues, gain, prominence, ability, effectively, opens, possibilities, advancement, especially, driven, perturbations, previous, synthesize, patches, genomenet, directpred, directly, sets, weights, typical, number, predict, point, polysemous, byol, excellent, benchmarks, imagenet, autoregressive, translate, texts, answer, questions, among, things, better, queries, bidirectional, transformers, suitable, developed, perform, build, supervising, combination, losses, kept, compressed, intrinsically, constitutes, because, needs, become, optimal, jargon, refers, setup, design, case, fully, contained, reused, combines, portion, similar, done, belongs, insofar, goal, classified, explicit, correlations, metadata, embedded, present, implicitly, autonomously, domain, operate, entirely, avoiding, pixel, level, focus, just, predicting, masked, visible, been, introduced, step, making, including, founded, included, enable, replicate, intellect, providing, exist, major, moves, beyond, views, preventing, collapse, rooted, includes, jea, invariant, sampling, dlvpm, generalizes, orthogonality, counterintuitively, converges, useful, minimum, reaching, trivial, solution, trivially, effective, extra, back, propagate, mathcal, mathrm, mathbb, frac, sum, infonce, optimize, jointly, nce, set, containing, proposal, distribution, minimizes, following, mid, ldots, clip, allows, pretraining, matching, have, span, angle, having, cosine, similarity, early, maximize, their, divided, those, match, identify, include, contain, both, minimize, presenting, possible, penalizes, reconstructed, minimizing, squared, comes, fact, essentially, associating, achieved, type, consist, maps, lower, reconstructs, decoder, specific, reproduce, words, tasked, captures, allowing, regenerate, many, pipelines, chosen, produces, sufficiently, confident, reducing, propagating, errors, instances, incorporated, refresh, evolve, understanding, emerging, patterns, existing, show, signs, aging, due, distributional, shifts, strategy, reduces, reliance, helping, maintain, performance, adaptive, also, plays, role, must, adapt, properties, change, over, scenarios, detect, incoming, deviates, previously, learned, behavior, generates, result, updating, retraining, becoming, outdated, enables, continuous, dynamic, environments, annotation, automatically, generated, assigns, widely, annotations, limited, unavailable, treating, surrogate, quantities, promising, years, found, practical, others, steps, first, solved, auxiliary, initialize, next, actual, performed, parameters, relying, externally, provided, within, solving, them, capturing, augmented, transformed, creates, serves, formulate, signal, involve, introducing, cropping, rotation, transformations, more, imitates, objects, outline, research, ijcai, iclr, neurips, ecml, pkdd, eccv, journals, conferences, topological, pac, occam, minimization, kernel, mathematical, foundations, roc, confusion, matrix, coefficient, determination, mechanistic, interpretability, loop, crowdsourcing, active, play, temporal, ecram, electrochemical, ram, memtransistor, spiking, physics, informed, radiance, deepdream, lenet, som, restricted, boltzmann, reservoir, esn, feedforward, isolation, outlier, ransac, markov, conditional, graphical, sdl, sne, pgd, pca, nmf, lda, ica, exploratory, shift, optics, dbscan, expectation, maximization, fuzzy, means, hierarchical, cure, birch, support, svm, relevance, rvm, logistic, naive, linear, bagging, ensembles, trees, apprenticeship, ontology, grammar, induction, rank, rules, automl, cleaning, density, modeling, problems, quantum, neuromorphic, curriculum, batch, meta, paradigms, mining, part, series, how, remove, message, please, removing, details, understandable, experts, improve, readers, confused, free, encyclopedia, item, printable, download, print, export, switch, legacy, parser, get, shortened, url, cite, permanent, link, here, actions, english, talk, tiếng, việt, українська, ไทย, português, polski, 한국어, 日本語, italiano, bahasa, indonesia, galego, français, فارسی, español, ελληνικά, deutsch, català, bosanski, العربية, subsection, top, personal, special, pages, community, portal, contribute, events, navigation, jump,
Text of the page (random words):
p learning decision trees ensembles bagging boosting random forest k nn linear regression naive bayes artificial neural networks logistic regression perceptron relevance vector machine rvm support vector machine svm clustering birch cure hierarchical k means fuzzy expectation maximization em dbscan optics mean shift dimensionality reduction factor analysis exploratory cca ica lda nmf pca pgd t sne sdl structured prediction graphical models bayes net conditional random field hidden markov anomaly detection ransac k nn local outlier factor isolation forest neural networks autoencoder deep learning feedforward neural network recurrent neural network lstm gru esn reservoir computing boltzmann machine restricted gan diffusion model som convolutional neural network u net lenet alexnet deepdream neural field neural radiance field physics informed neural networks transformer vision mamba spiking neural network memtransistor electrochemical ram ecram reinforcement learning q learning policy gradient sarsa temporal difference td multi agent self play learning with humans active learning crowdsourcing human in the loop mechanistic interpretability rlhf model diagnostics coefficient of determination confusion matrix learning curve roc curve mathematical foundations kernel machines bias variance tradeoff computational learning theory empirical risk minimization occam learning pac learning statistical learning vc theory topological deep learning journals and conferences aaai cvpr eccv ecml pkdd emnlp iccv neurips icml iclr ijcai ml jmlr related articles glossary of artificial intelligence list of datasets for machine learning research list of datasets in computer vision and image processing outline of machine learning v t e self supervised learning ssl is a paradigm in machine learning where a model is trained on a task using the data itself to generate supervisory signals rather than relying on externally provided labels in the context of neural networks self supervised learning aims to leverage inherent structures or relationships within the input data to create meaningful training signals ssl tasks are designed so that solving them requires capturing essential features or relationships in the data the input data is typically augmented or transformed in a way that creates pairs of related samples where one sample serves as the input and the other is used to formulate the supervisory signal this augmentation can involve introducing noise cropping rotation or other transformations self supervised learning more closely imitates the way humans learn to classify objects 1 during ssl the model learns in two steps first the task is solved based on an auxiliary or pretext classification task using pseudo labels which help to initialize the model parameters 2 3 next the actual task is performed with supervised or unsupervised learning 4 5 6 self supervised learning has produced promising results in recent years and has found practical application in fields such as audio processing and is being used by facebook and others for speech recognition 7 pseudo labels edit pseudo labels are automatically generated labels that a model assigns to unlabeled data based on its own predictions they are widely used in self supervised and semi supervised learning where ground truth annotations are limited or unavailable by treating predicted labels as surrogate ground truth learning algorithms can make use of large quantities of unlabeled data in the training process 8 pseudo labeling also plays an important role in systems that must adapt to concept drift where the statistical properties of the data change over time in these scenarios the model may detect that an incoming instance deviates from previously learned behavior the system then generates a classification result for that instance and this predicted class is used as a pseudo label for updating or retraining model components that are becoming outdated this approach enables continuous adaptation in dynamic environments without requiring manual annotation 9 10 in many adaptive learning pipelines pseudo labels are chosen when the classifier produces sufficiently confident predictions reducing the risk of propagating errors these pseudo labeled instances are then incorporated into training to refresh or evolve the model s understanding of emerging data patterns particularly when existing components show signs of aging due to drift or distributional shifts this strategy reduces reliance on manual labeling while helping maintain long term model performance types edit autoassociative self supervised learning edit autoassociative self supervised learning is a specific category of self supervised learning where a neural network is trained to reproduce or reconstruct its own input data 11 in other words the model is tasked with learning a representation of the data that captures its essential features or structure allowing it to regenerate the original input the term autoassociative comes from the fact that the model is essentially associating the input data with itself this is often achieved using autoencoders which are a type of neural network architecture used for representation learning autoencoders consist of an encoder network that maps the input data to a lower dimensional representation latent space and a decoder network that reconstructs the input from this representation the training process involves presenting the model with input data and requiring it to reconstruct the same data as closely as possible the loss function used during training typically penalizes the difference between the original input and the reconstructed output e g mean squared error by minimizing this reconstruction error the autoencoder learns a meaningful representation of the data in its latent space contrastive self supervised learning edit for a binary classification task training data can be divided into positive examples and negative examples positive examples are those that match the target for example if training a classifier to identify birds the positive training data would include images that contain birds negative examples would be images that do not 12 contrastive self supervised learning uses both positive and negative examples the loss function in contrastive learning is used to minimize the distance between positive sample pairs while maximizing the distance between negative sample pairs 12 an early example uses a pair of 1 dimensional convolutional neural networks to process a pair of images and maximize their agreement 13 contrastive language image pre training clip allows joint pretraining of a text encoder and an image encoder such that a matching image text pair have image encoding vector and text encoding vector that span a small angle having a large cosine similarity infonce noise contrastive estimation 14 is a method to optimize two models jointly based on noise contrastive estimation nce 15 given a set x x 1 x n displaystyle x left x_ 1 ldots x_ n right of n displaystyle n random samples containing one positive sample from p x t k c t displaystyle p left x_ t k mid c_ t right and n 1 displaystyle n 1 negative samples from the proposal distribution p x t k displaystyle p left x_ t k right it minimizes the following loss function l n e x log f k x t k c t x j x f k x j c t displaystyle mathcal l _ mathrm n mathbb e _ x left log frac f_ k left x_ t k c_ t right sum _ x_ j in x f_ k left x_ j c_ t right right non contrastive self supervised learning edit non contrastive self supervised learning ncssl uses only positive examples counterintuitively ncssl converges on a useful local minimum rather than reaching a trivial solution with zero loss for the example of binary classification it would trivially learn to classify each example as positive effective ncssl requires an extra predictor on the online side that does not back propagate on the target side 12 joint embedding and predictive architectures edit a major class of self supervised learning moves beyond contrastive pairs instead maximizing the agreement between views while preventing collapse through statistical constraints rooted in deep canonical correlation analysis deep cca 16 this approach includes joint embedding architectures jea like barlow twins 17 and vicreg 18 which enforce covariance constraints to learn invariant representations without negative sampling deep latent variable path modelling dlvpm 19 generalizes this to multimodal systems using path models to enforce correlation and orthogonality across diverse data types in 2022 yann lecun introduced joint embedding predictive architectures jepa as a step towards decision making reasoning and autonomous human intelligence in machines including self improvement through autonomous learning founded in representation learning lecun included the concept of a world model in jepa which aims to enable machines to replicate human intellect by providing machines with a concept for the world in which they exist 20 unlike autoencoders jepas operate entirely in latent space avoiding pixel level noise to focus on semantic structure rather than just learning invariance jepas learn by predicting masked latent representations from visible context 21 jepa has been applied to domains such as image analysis 22 audio processing 23 and motion in images and video 24 comparison with other forms of machine learning edit ssl belongs to supervised learning methods insofar as the goal is to generate a classified output from the input at the same time however it does not require the explicit use of labeled input output pairs instead correlations metadata embedded in the data or domain knowledge present in the input are implicitly and autonomously extracted from the data these supervisory signals extracted from the data can then be used for training 1 ssl is similar to unsupervised learning in that it does not require labels in the sample data unlike unsupervised learning however learning is not done using inherent data structures semi supervised learning combines supervised and unsupervised learning requiring only a small portion of the learning data be labeled 3 in transfer learning a model designed for one task is reused on a different task 25 training an autoencoder intrinsically constitutes a self supervised process because the output pattern needs to become an optimal reconstruction of the input pattern itself however in current jargon the term self supervised often refers to tasks based on a pretext task training setup this involves the human design of such pretext task s unlike the case of fully self contained autoencoder training 11 in reinforcement learning self supervising learning from a combination of losses can create abstract representations where only the most important information about the state are kept in a compressed way 26 examples edit self supervised learning is particularly suitable for speech recognition for example facebook developed wav2vec a self supervised algorithm to perform speech recognition using two deep convolutional neural networks that build on each other 7 google s bidirectional encoder representations from transformers bert model is used to better understand the context of search queries 27 openai s gpt 3 is an autoregressive language model that can be used in language processing it can be used to translate texts or answer questions among other things 28 bootstrap your own latent byol is a ncssl that produced excellent results on imagenet and on transfer and semi supervised benchmarks 29 the yarowsky algorithm is an example of self supervised learning in natural language processing from a small number of labeled examples it learns to predict which word sense of a polysemous word is being used at a given point in text directpred is a ncssl that directly sets the predictor weights instead of learning it via typical gradient descent 12 self genomenet is an example of self supervised learning in genomics 30 selfapr is an example of self supervised learning applied to automated program repair where a neural model is trained on perturbations of a previous version of the program under repair and uses test execution diagnostics to synthesize patches 31 self supervised learning continues to gain prominence as a new approach across diverse fields its ability to leverage unlabeled data effectively opens new possibilities for advancement in machine learning especially in data driven application domains references edit 1 2 bouchard louis 25 november 2020 what is self supervised learning will machines ever be able to learn like humans medium retrieved 9 june 2021 doersch carl zisserman andrew october 2017 multi task self supervised visual learning 2017 ieee international conference on computer vision iccv ieee pp 2070 2079 arxiv 1708 07860 doi 10 1109 iccv 2017 226 isbn 978 1 5386 1032 9 s2cid 473729 1 2 beyer lucas zhai xiaohua oliver avital kolesnikov alexander october 2019 s4l self supervised semi supervised learning 2019 ieee cvf international conference on computer vision iccv ieee pp 1476 1485 arxiv 1905 03670 doi 10 1109 iccv 2019 00156 isbn 978 1 7281 4803 8 s2cid 167209887 doersch carl gupta abhinav efros alexei a december 2015 unsupervised visual representation learning by context prediction 2015 ieee international conference on computer vision iccv ieee pp 1422 1430 arxiv 1505 05192 doi 10 1109 iccv 2015 167 isbn 978 1 4673 8391 2 s2cid 9062671 zheng xin wang yong wang guoyou liu jianguo april 2018 fast and robust segmentation of white blood cell images by self supervised learning micron 107 55 71 doi 10 1016 j micron 2018 01 010 issn 0968 4328 pmid 29425969 s2cid 3796689 gidaris spyros bursuc andrei komodakis nikos perez patrick perez cord matthieu october 2019 boosting few shot visual learning with self supervision 2019 ieee cvf international conference on computer vision iccv ieee pp 8058 8067 arxiv 1906 05186 doi 10 1109 iccv 2019 00815 isbn 978 1 7281 4803 8 s2cid 186206588 1 2 wav2vec state of the art speech recognition through self supervision ai facebook com retrieved 9 june 2021 lee dong hyun 2013 pseudo label the simple and efficient semi supervised learning method for deep neural networks icml 2013 workshop on challenges in representation learning wrepl gama joão 2014 a survey on concept drift adaptation acm computing surveys 46 4 1 37 doi 10 1145 2523813 xu k li y deng r chen k xu j 2019 self evolving android malware detection system ieee european symposium on security and privacy euros p pp 47 62 doi 10 1109 eurosp 2019 00014 1 2 kramer mark a 1991 nonlinear principal component analysis using autoassociative neural networks pdf aiche journal 37 2 233 243 bibcode 1991aiche 37 233k doi 10 1002 aic 690370209 1 2 3 4 demystifying a key self supervised learning technique non contrastive learning ai facebook com retrieved 5 october 2021 becker suzanna hinton geoffrey e january 1992 self organizing neural network that discovers surfaces in random dot stereograms natur...
|
