Meta tags:
author= Baifeng Shi;
Headings (most frequently used words):
-
Text of the page (most frequently used words):
and (27), the (14), pre (12), training (12), vision (11), models (11), visual (10), resolution (10), with (10), hosted (9), video (9), baifeng (9), shi (9), scaling (8), pdf (8), that (8), nbsp (8), website (7), abstract (7), trevor (7), darrell (7), high (7), 2025 (6), 2026 (6), university (5), slides (5), 2024 (5), next (5), han (5), this (5), vit (5), code (5), nvila (5), top (4), attention (4), apr (4), autogaze (4), cvpr (4), efficient (4), understanding (4), learning (4), from (4), videos (4), grasping (4), object (4), model (4), larger (4), long (4), jun (3), 2023 (3), down (3), what (3), wang (3), pixels (3), cheng (3), before (3), scalable (3), via (3), jitendra (3), malik (3), trained (3), toys (3), can (3), find (3), representations (3), image (3), are (3), usually (3), efficiency (3), vila (3), then (3), benchmarks (3), song (3), hongxu (3), danny (3), yin (3), jan (3), kautz (3), pavlo (3), molchanov (3), liu (3), ps3 (3), res (3), benchmark (3), mllms (3), principles (2), applications (2), bottom (2), peking (2), ding (2), prof (2), berkeley (2), ziyang (2), google (2), zhu (2), chen (2), may (2), watching (2), billion (2), once (2), attend (2), autoregressive (2), gazing (2), make (2), sensorimotor (2), ilija (2), radosavovic (2), humanoid (2), locomotion (2), token (2), prediction (2), enables (2), data (2), such (2), zhang (2), show (2), policy (2), randomly (2), constructed (2), directly (2), zero (2), shot (2), generalize (2), real (2), world (2), success (2), rates (2), key (2), level (2), generalization (2), lies (2), centric (2), physical (2), intelligence (2), generalist (2), robotic (2), selected (2), publications (2), scales (2), better (2), than (2), also (2), not (2), open (2), vlms (2), accuracy (2), its (2), resolutions (2), tokens (2), scale (2), efficiently (2), process (2), images (2), fine (2), tuning (2), reduces (2), latency (2), demo (2), yao (2), sifei (2), previous (2), performance (2), propose (2), mllm (2), conference (2), highlight (2), boyi (2), redundant (2), form (2), david (2), degree (2), techbeat, oct, yufei, mar, group, vgg, oxford, guanqi, zhan, tea, talk, singapore, kai, deepmind, tengda, princeton, olga, russakovsky, tyler, xindi, boston, bryan, plummer, tianle, baai, cohere, simão, herdade, gaze, workshop, yihua, invited, talks, imitation, easier, mae, sequences, oral, presentation, corl, ken, goldberg, letian, robot, formulate, problem, walk, wild, youtube, spotlight, neurips, koushil, sreenath, sarthak, kamat, jathushan, rajasegaran, bike, arxiv, roei, herzig, shankar, shastry, anirudh, pai, konstantinos, kallidromitis, henry, tsai, haoru, xue, matteo, gioia, rachel, yuvan, sharma, dantong, niu, grasp, anything, playing, random, preprint, steerable, foundation, emergent, capabilities, robotics, smaller, run, learn, similar, eccv, xin, maolin, mao, when, need, family, designed, optimize, both, for, multi, building, improve, architecture, first, spatial, temporal, compressing, compress, approach, conduct, systematic, investigation, enhance, throughout, entire, lifecycle, deployment, matches, surpasses, many, leading, proprietary, across, wide, range, same, time, costs, memory, usage, filling, decoding, our, available, facilitate, reproducibility, xiaolong, daguang, ranjay, krishna, jinyi, vishwesh, nath, chieh, huang, hsieh, yukang, yunhao, fang, xiuyu, dacheng, yuxian, shiyi, cao, haocheng, shang, yang, yuming, lou, zhuoyang, ligeng, zhijian, frontier, language, siglip, dinov2, low, 384x384, limiting, their, perception, near, constant, cost, processes, prompt, aware, patch, selection, introduce, state, art, encoder, achieves, various, including, 4kpro, introduced, work, marco, pavone, cai, most, but, current, every, single, pixel, despite, reduncy, which, extremely, inefficient, fps, super, light, weight, automatically, removes, patches, processing, 100, accelerates, vits, enabling, frame, achieve, superior, results, especially, proposed, hlvid, chan, aaron, reite, eigen, hanrong, lian, stephanie, wechat, github, scholar, email, build, member, technical, staff, previously, earned, advised,
Text of the page (random words):
baifeng shi baifeng shi i am a member of technical staff at physical intelligence i previously earned my ph d degree from uc berkeley advised by prof trevor darrell and a b s degree from peking university i build generalist vision and robotic models email nbsp nbsp google scholar nbsp nbsp github nbsp nbsp cv nbsp nbsp wechat selected publications vision attend before attention efficient and scalable video understanding via autoregressive gazing baifeng shi stephanie fu long lian hanrong ye david eigen aaron reite boyi li jan kautz song han david m chan pavlo molchanov trevor darrell hongxu danny yin cvpr 2026 conference highlight abstract website pdf code models data hlvid benchmark demo most pixels in a video are redundant but current mllms usually process every single pixel of an video despite the reduncy which is extremely inefficient and not scalable to high resolution high fps or long form videos we propose autogaze a super light weight model that automatically removes the redundant patches in a video before processing it with a vit or mllm autogaze reduces visual tokens by 4 100 and accelerates vits and mllms by up to 19 enabling mllms to scale to 1k frame 4k resolution videos and achieve superior results on video benchmarks especially on the proposed high resolution long form video benchmark scaling vision pre training to 4k resolution baifeng shi boyi li han cai yao lu sifei liu marco pavone jan kautz song han trevor darrell pavlo molchanov hongxu danny yin cvpr 2025 conference highlight abstract website pdf code previous vision models such as siglip and dinov2 are usually pre trained at low resolutions e g 384x384 limiting their performance of high resolution perception we propose ps3 a vision model that scales pre training to 4k resolution with a near constant cost ps3 efficiently processes high res images via top down i e prompt aware patch selection we then introduce vila hd a state of the art high res mllm with ps3 as the vision encoder vila hd achieves better performance and efficiency than previous models on various benchmarks including a 4k res benchmark 4kpro that s introduced in this work nvila efficient frontier visual language models zhijian liu ligeng zhu baifeng shi zhuoyang zhang yuming lou shang yang haocheng xi shiyi cao yuxian gu dacheng li xiuyu li yunhao fang yukang chen cheng yu hsieh de an huang an chieh cheng vishwesh nath jinyi hu sifei liu ranjay krishna daguang xu xiaolong wang pavlo molchanov jan kautz hongxu danny yin song han yao lu cvpr 2025 abstract website demo pdf code models nvila is a family of open vlms designed to optimize both efficiency and accuracy for efficient video understanding and multi image understanding building on top of vila we improve its model architecture by first scaling up the spatial and temporal resolutions and then compressing visual tokens this scale then compress approach enables nvila to efficiently process high resolution images and long videos we also conduct a systematic investigation to enhance the efficiency of nvila throughout its entire lifecycle from training and fine tuning to deployment nvila matches or surpasses the accuracy of many leading open and proprietary vlms across a wide range of image and video benchmarks at the same time it reduces training costs by 4 5 fine tuning memory usage by 3 4 pre filling latency by 1 6 2 2 and decoding latency by 1 2 2 8 we make our code and models available to facilitate reproducibility when do we not need larger vision models baifeng shi ziyang wu maolin mao xin wang trevor darrell eccv 2024 abstract pdf code we find that smaller vision models e g vit b or vit l run on larger image scales are usually better than larger models e g vit h vit g and can also learn similar representations as larger models selected publications robotics π 0 7 a steerable generalist robotic foundation model with emergent capabilities physical intelligence preprint 2026 pdf website we show that a grasping policy trained on randomly constructed toys can directly zero shot generalize to real world object grasping with 67 80 success rates we find that the key to this level of generalization lies in the object centric visual representations learning to grasp anything by playing with random toys dantong niu yuvan sharma baifeng shi rachel ding matteo gioia haoru xue henry tsai konstantinos kallidromitis anirudh pai shankar s shastry trevor darrell jitendra malik roei herzig arxiv 2025 abstract pdf website we show that a grasping policy trained on randomly constructed toys can directly zero shot generalize to real world object grasping with 67 80 success rates we find that the key to this level of generalization lies in the object centric visual representations humanoid locomotion as next token prediction ilija radosavovic bike zhang baifeng shi jathushan rajasegaran sarthak kamat trevor darrell koushil sreenath jitendra malik neurips 2024 spotlight abstract pdf website we formulate humanoid locomotion as a next token prediction problem this enables learning to walk from in the wild data such as youtube videos robot learning with sensorimotor pre training ilija radosavovic baifeng shi letian fu ken goldberg trevor darrell jitendra malik corl 2023 oral presentation abstract pdf website we make imitation learning easier by mae pre training on sensorimotor sequences invited talks jun 2026 attend before attention efficient and scalable video understanding via autoregressive gazing gaze workshop cvpr 2026 hosted by yihua cheng may 2026 watching 10 billion pixels at once with autogaze cohere hosted by simão herdade slides may 2026 watching 10 billion pixels at once with autogaze baai apr 2025 scaling vision pre training to 4k resolution boston university hosted by tianle chen and bryan plummer slides apr 2025 scaling vision pre training to 4k resolution princeton university hosted by xindi wu tyler zhu and olga russakovsky slides apr 2025 scaling vision pre training to 4k resolution google deepmind hosted by tengda han jun 2024 scaling up visual pre training what s next ai tea talk singapore hosted by kai wang apr 2024 scaling up visual pre training what s next vgg university of oxford hosted by guanqi zhan slides mar 2024 scaling up visual pre training what s next prof yi ma s group uc berkeley hosted by ziyang wu oct 2023 principles and applications of bottom up and top down visual attention peking university hosted by yufei ding slides jun 2023 principles and applications of bottom up and top down visual attention techbeat
|
