Meta tags:
author=Jan Hendrik Metzen;
description=Personal website of Jan Hendrik Metzen, a research scientist working on tabular foundation models, scalable pretraining, efficient architectures, and reliable machine learning. ;
keywords=tabular foundation models, machine learning, scalable pretraining, efficient architectures, AutoML, robust AI;

Headings (most frequently used words):

jan, hendrik, metzen, news, selected, publications, 2026, 2024, 2023, 2022, research, interests,

Text of the page (most frequently used words):
and (61), the (38), models (26), for (22), image (19), neural (14), that (12), with (11), can (11), vision (11), #research (11), foundation (11), jan (10), hendrik (10), metzen (10), adversarial (10), from (9), architectures (9), this (9), classifiers (9), subgroups (9), search (9), work (9), free (9), pretraining (9), architecture (9), 2026 (8), which (8), tabular (8), training (7), learning (7), shot (7), semantic (7), inference (7), datasets (7), efficient (7), attention (6), patch (6), pdf (6), arxiv (6), computer (6), performance (6), data (6), systematic (6), errors (6), zero (6), conditioning (6), language (6), class (6), autoclip (6), tokenizer (6), these (5), are (5), show (5), model (5), has (5), prompt (5), 2024 (5), machine (5), large (5), templates (5), tabpfn (5), scalable (5), also (5), our (5), based (4), such (4), their (4), how (4), attacks (4), under (4), robust (4), abs (4), both (4), rare (4), label (4), time (4), prior (4), descriptors (4), hat (4), scaling (4), scale (4), reliable (4), about (3), all (3), source (3), vulnerability (3), when (3), control (3), propose (3), novel (3), patches (3), classification (3), object (3), low (3), than (3), conference (3), robustness (3), substantially (3), domain (3), space (3), where (3), promptattack (3), synthetic (3), images (3), spatial (3), synthesis (3), tasks (3), across (3), broad (3), range (3), studied (3), tuning (3), publications (3), scientist (3), working (3), labs (3), aleph (3), alpha (3), adaptation (3), one (3), million (3), rows (3), design (3), make (3), powered (2), github (2), you (2), way (2), recognition (2), main (2), reasoning (2), scene (2), scaled (2), dot (2), product (2), major (2), provide (2), understanding (2), key (2), token (2), target (2), detr (2), find (2), imagenet (2), 2022 (2), chaithanya (2), kumar (2), mummadi (2), despite (2), semantically (2), well (2), shift (2), identify (2), very (2), recent (2), advances (2), text (2), descriptions (2), synthesized (2), accuracy (2), 2023 (2), transactions (2), improves (2), shown (2), pre (2), trained (2), diffusion (2), methods (2), content (2), using (2), edges (2), expensive (2), manual (2), segmentation (2), new (2), fine (2), uses (2), layouts (2), they (2), semantics (2), via (2), better (2), effectively (2), built (2), have (2), automatically (2), descriptor (2), words (2), encoded (2), its (2), auto (2), tunes (2), weights (2), few (2), philipp (2), technical (2), report (2), next (2), generation (2), contributed (2), family (2), hugging (2), face (2), developed (2), area (2), includes (2), more (2), networks (2), hardware (2), process (2), improving (2), efficiency (2), survey (2), automl (2), worked (2), universal (2), transformer (2), optimization (2), hierarchical (2), while (2), end (2), learn (2), current (2), interests (2), curriculum (2), vitae (2), open (2), was (2), senior (2), copyright, theme, hosted, photos, last, updated, july, unsplash, pages, folio, jekyll, even, add, little, note, best, reach, transformers, revolutionizing, benefit, allows, parts, jointly, paper, global, confronted, theoretical, relate, adversary, ability, misdirect, queries, single, objectives, crafting, explicitly, effectiveness, proposed, popular, vits, deits, detection, occupying, input, lead, accuracies, vit, reduce, map, coco, less, proceedings, ieee, cvf, pattern, giulio, lovisotto, nicole, finnie, mauricio, munoz, give, your, considered, harmful, excellent, average, case, many, deteriorate, coherent, were, represented, impact, fairness, demographic, minority, groups, safety, challenge, subpar, not, annotated, occurrence, leverage, textual, prompts, conditioned, tackle, exponentially, growing, number, employ, combinatorial, testing, denote, procedure, interpreted, attack, study, subgroup, coverage, identifiability, controlled, setting, identifies, high, thereupon, apply, international, robin, hutmacher, grace, hua, valentyn, boreiko, dan, zhang, identification, niclas, popp, matthias, hein, feature, distillation, transfer, great, progress, integrating, existing, describe, hand, crafted, inputs, either, ambiguous, require, annotations, address, limitations, enable, grained, introduce, concept, emphneural, extracted, advantageous, rich, desired, containing, detailed, geometry, experimentally, achieve, similar, superior, pixel, level, alignment, classes, compared, those, created, maps, same, capture, instance, separation, orientation, other, options, depth, moreover, generated, layout, augment, real, various, perception, european, eccv, jiayi, wang, kevin, alexander, laube, yumeng, shin, cheng, julio, borges, anna, khoreva, upon, clip, remarkable, different, ways, creating, sets, every, ranging, manually, engineered, over, obtained, random, characters, until, now, deriving, respective, remained, nearly, unchanged, classify, maximizes, cosine, similarity, between, averaged, encoding, however, weighing, equally, suboptimal, certain, match, visual, clues, given, others, method, per, each, template, statistics, similarities, fully, unsupervised, computational, overhead, easily, implemented, lines, code, outperforms, baselines, consistently, percent, point, piyapat, saranrittichai, léo, grinsztajn, klemens, flöge, oscar, felix, birkel, jund, brendan, roof, mihir, manium, shi, bin, hoo, magnus, bühler, anurag, garg, dominik, safaric, jake, robertson, benjamin, jäger, simone, alessi, adrian, hayler, vladyslav, moroshan, lennart, purucker, singer, alan, arazi, julien, siems, georg, grab, nick, erickson, siyuan, guo, eliott, kalfon, simon, bing, david, salinas, clara, cornu, lilly, charlotte, wehrhahn, diana, kriuchkova, kursat, kaya, lydia, sidhoum, marie, salmon, jerry, chen, madelon, hulsebos, yann, lecun, samuel, müller, bernhard, schölkopf, sauraj, gambhir, noah, hollmann, frank, hutter, selected, launched, tool, searching, generating, customizing, vector, graphics, svgstud, aug, joined, feb, part, previous, role, presenting, available, llms, liberated, static, vocabularies, mar, released, may, news, domains, collected, practical, lessons, making, effective, bag, tricks, meta, multi, objective, nas, lamarckian, evolution, through, network, morphisms, vast, diversity, difficult, aware, automate, reducing, cost, development, overview, see, dense, prediction, certifiably, against, shape, biased, representations, affect, common, corruptions, test, perturbations, fail, them, safer, used, generative, uncover, identified, vulnerabilities, detectors, interested, encoder, decoder, operate, bytes, backbone, operates, regex, defined, approach, billion, parameters, building, learns, sequence, aggregation, instead, relying, predefined, word, boundaries, sombrero, autoregressive, structured, diverse, during, then, fit, entirely, context, without, turning, fitting, into, fast, direct, focus, larger, behavior, extends, google, scholar, live, böblingen, freiburg, outside, enjoy, spending, wife, three, children, always, happy, connect, people, contact, broader, span, software, member, chair, including, probability, calibration, kernel, ridge, gaussian, modules, scikit, neurips, eliza, industrial, fellow, ellis, previously, researcher, team, llm, before, expert, centered, bosch, center, artificial, intelligence, develop, focuses, laws, contribute, scales, accelerating, projects, toggle, navigation, skip,

Text of the page (random words):
jan hendrik metzen skip to main content jan hendrik metzen toggle navigation about current publications projects curriculum vitae jan hendrik metzen research scientist at prior labs tabular foundation models scalable pretraining i am a research scientist at prior labs where i develop the next generation of tabular foundation models my work focuses on scalable pretraining novel architectures and scaling laws i contribute to tabpfn 3 which scales tabular foundation models to datasets with up to one million training rows while substantially accelerating training and inference previously i was a senior ai researcher in aleph alpha research s foundation models team working on efficient llm pretraining tokenizer free architectures and large scale optimization before that i was a senior expert at the bosch center for artificial intelligence where my research centered on robust and reliable computer vision neural architecture search and synthetic data my broader interests span efficient and reliable machine learning automl and open source software i am an ellis member an eliza industrial fellow and an area chair for neurips 2026 i have also contributed to scikit learn including its probability calibration kernel ridge and gaussian process modules i live in böblingen and work in freiburg outside research i enjoy spending time with my wife and our three children i am always happy to connect with people working on tabular foundation models scalable pretraining efficient or tokenizer free architectures and open source machine learning contact me publications curriculum vitae google scholar github tabpfn 3 tokenizer free hat models research interests tabular foundation models and scalable pretraining i work on foundation models for structured data that learn from diverse synthetic datasets during pretraining and can then fit entirely new datasets in context without fine tuning effectively turning model fitting into fast direct inference my current focus is scaling these models to substantially larger datasets understanding their scaling behavior and improving the efficiency of both pretraining and inference this work includes tabpfn 3 which extends tabular foundation models to datasets with up to one million training rows efficient and tokenizer free architectures i am interested in architectures that make large scale pretraining and inference more efficient at aleph alpha research i worked on tokenizer free language models domain adaptation and large scale optimization our hierarchical autoregressive transformer hat uses a hierarchical architecture in which the encoder and decoder operate on bytes while the backbone operates on regex defined words we scaled this approach to models with up to 70 billion parameters building on hat sombrero learns the sequence aggregation end to end instead of relying on predefined word boundaries reliable and robust machine learning my work on reliable computer vision studied both how models fail and how to make them safer we used generative models to uncover systematic errors of image classifiers on rare subgroups and systematic errors of object detectors and identified vulnerabilities of transformer based networks to adversarial patch and token attacks we developed architectures that are certifiably robust against patch attacks for image classification and semantic segmentation as well as adversarial training methods for universal perturbations and universal adversarial patches i also worked on test time adaptation under domain shift and studied how shape biased representations affect robustness to common image corruptions automl and neural architecture search the vast design space of neural networks and the diversity of inference hardware make manual architecture design difficult to scale hardware aware neural architecture search can automate this process improving design efficiency and reducing the cost of ai development for an overview see our survey on neural architecture search and our survey on neural architecture search for dense prediction tasks in computer vision my work in this area also includes efficient architecture search through network morphisms multi objective nas via lamarckian evolution and meta learning neural architectures for few shot learning we also collected practical lessons for making search more effective in our bag of tricks for neural architecture search i also co developed autoclip which automatically tunes zero shot classifiers for vision language models and improves performance across a broad range of domains news may 13 2026 we released the tabpfn 3 technical report and model weights on hugging face scaling tabular foundation models to datasets with up to one million training rows mar 16 2026 as part of my previous role at aleph alpha research i contributed to a family of llms liberated from static vocabularies presenting tokenizer free architectures for efficient pretraining adaptation and inference the hat model family is available on hugging face feb 1 2026 i joined prior labs as a research scientist working on the next generation of tabular foundation models aug 12 2024 i launched svgstud io an ai powered tool for searching generating and customizing scalable vector graphics selected publications 2026 tabpfn 3 technical report léo grinsztajn klemens flöge oscar key felix birkel philipp jund brendan roof mihir manium shi bin hoo magnus bühler anurag garg dominik safaric jake robertson benjamin jäger simone alessi adrian hayler vladyslav moroshan lennart purucker philipp singer alan arazi julien siems jan hendrik metzen georg grab nick erickson siyuan guo eliott kalfon simon bing david salinas clara cornu lilly charlotte wehrhahn diana kriuchkova kursat kaya lydia sidhoum marie salmon jerry chen madelon hulsebos yann lecun samuel müller bernhard schölkopf sauraj gambhir noah hollmann and frank hutter 2026 arxiv pdf models 2024 autoclip auto tuning zero shot classifiers for vision language models jan hendrik metzen piyapat saranrittichai and chaithanya kumar mummadi transactions on machine learning research 2024 abs arxiv pdf classifiers built upon vision language models such as clip have shown remarkable zero shot performance across a broad range of image classification tasks prior work has studied different ways of automatically creating descriptor sets for every class based on prompt templates ranging from manually engineered templates over templates obtained from a large language model to templates built from random words and characters up until now deriving zero shot classifiers from the respective encoded class descriptors has remained nearly unchanged i e classify to the class that maximizes cosine similarity between its averaged encoded class descriptors and the image encoding however weighing all class descriptors equally can be suboptimal when certain descriptors match visual clues on a given image better than others in this work we propose autoclip a method for auto tuning zero shot classifiers autoclip tunes per image weights to each prompt template at inference time based on statistics of class descriptor image similarities autoclip is fully unsupervised has very low computational overhead and can be easily implemented in few lines of code we show that autoclip outperforms baselines across a broad range of vision language models datasets and prompt templates consistently and by up to 3 percent point accuracy label free neural semantic image synthesis jiayi wang kevin alexander laube yumeng li jan hendrik metzen shin i cheng julio borges and anna khoreva in european conference on computer vision eccv 2024 abs arxiv pdf recent work has shown great progress in integrating spatial conditioning to control large pre trained text to image diffusion models despite these advances existing methods describe the spatial image content using hand crafted conditioning inputs which are either semantically ambiguous e g edges or require expensive manual annotations e g semantic segmentation to address these limitations we propose a new label free way of conditioning diffusion models to enable fine grained spatial control we introduce the concept of emphneural semantic image synthesis which uses neural layouts extracted from pre trained foundation models as conditioning neural layouts are advantageous as they provide rich descriptions of the desired image containing both semantics and detailed geometry of the scene we experimentally show that images synthesized via neural semantic image synthesis achieve similar or superior pixel level alignment of semantic classes compared to those created using expensive semantic label maps at the same time they capture better semantics instance separation and object orientation than other label free conditioning options such as edges or depth moreover we show that images generated by neural layout conditioning can effectively augment real data for training various perception tasks feature distillation improves zero shot transfer from synthetic images niclas popp jan hendrik metzen and matthias hein transactions on machine learning research 2024 arxiv pdf 2023 identification of systematic errors of image classifiers on rare subgroups jan hendrik metzen robin hutmacher n grace hua valentyn boreiko and dan zhang in international conference on computer vision 2023 abs arxiv pdf despite excellent average case performance of many image classifiers their performance can substantially deteriorate on semantically coherent subgroups of the data that were under represented in the training data these systematic errors can impact both fairness for demographic minority groups as well as robustness and safety under domain shift a major challenge is to identify such subgroups with subpar performance when the subgroups are not annotated and their occurrence is very rare we leverage recent advances in text to image models and search in the space of textual descriptions of subgroups prompts for subgroups where the target model has low performance on the prompt conditioned synthesized data to tackle the exponentially growing number of subgroups we employ combinatorial testing we denote this procedure as promptattack as it can be interpreted as an adversarial attack in a prompt space we study subgroup coverage and identifiability with promptattack in a controlled setting and find that it identifies systematic errors with high accuracy thereupon we apply promptattack to imagenet classifiers and identify novel systematic errors on rare subgroups 2022 give me your attention dot product attention considered harmful for adversarial patch robustness giulio lovisotto nicole finnie mauricio munoz chaithanya kumar mummadi and jan hendrik metzen in proceedings of the ieee cvf conference on computer vision and pattern recognition 2022 abs arxiv pdf neural architectures based on attention such as vision transformers are revolutionizing image recognition their main benefit is that attention allows reasoning about all parts of a scene jointly in this paper we show how the global reasoning of scaled dot product attention can be the source of a major vulnerability when confronted with adversarial patch attacks we provide a theoretical understanding of this vulnerability and relate it to an adversary s ability to misdirect the attention of all queries to a single key token under the control of the adversarial patch we propose novel adversarial objectives for crafting adversarial patches which target this vulnerability explicitly we show the effectiveness of the proposed patch attacks on popular image classification vits and deits and object detection models detr we find that adversarial patches occupying 0 5 of the input can lead to robust accuracies as low as 0 for vit on imagenet and reduce the map of detr on ms coco to less than 3 you can even add a little note about which of these is the best way to reach you copyright 2026 jan hendrik metzen powered by jekyll with al folio theme hosted by github pages photos from unsplash last updated july 05 2026