Meta tags:

Headings (most frequently used words):

depth, with, lidar, is, high, and, the, accurate, metric, estimation, can, but, anything, resolution, comparisons, monocular, methods, application, street, grasping, res, even, cost, low, prompting, for, 4k, abstract, arkit, more, testing, results, reconstruction, generalized, robotic, acknowledgements, citation, prompt, method, following, highlights, generate, maps, struggle, consistent, scale, information, after, aligning, dense, preferred, its, noisy, due, to, limited, power, prompts, be, replaced, vehicle, achieving, precision, in, scenes, if, policy, trained, only, on, diffuse, objects, our, help, grasp, transparent, specular, ones, outperforming, rgb,

Text of the page (most frequently used words):
depth (29), the (28), and (24), for (13), lidar (13), #anything (7), metric (7), with (7), prompt (7), #resolution (6), accurate (6), prompting (5), estimation (5), data (5), our (5), peng (4), robot (4), can (4), grasping (4), high (4), low (4), foundation (4), models (4), thanks (3), support (3), including (3), model (3), video (3), generalized (3), robotic (3), reconstruction (3), arkit (3), cost (3), but (3), power (3), useful (2), add (2), ons (2), lin (2), haotong (2), sida (2), chen (2), jingxiao (2), songyou (2), sun (2), jiaming (2), liu (2), minghuan (2), bao (2), hujun (2), feng (2), jiashi (2), zhou (2), xiaowei (2), kang (2), bingyi (2), arxiv (2), thank (2), objects (2), help (2), even (2), rgb (2), your (2), browser (2), does (2), not (2), tag (2), application (2), prompts (2), achieving (2), street (2), more (2), results (2), iphone (2), res (2), limited (2), comparisons (2), monocular (2), methods (2), new (2), use (2), approach (2), that (2), datasets (2), scalable (2), pipeline (2), scannet (2), benefits (2), downstream (2), applications (2), method (2), dataset (2), university (2), website, template, viser, nerfies, lior, yariv, inproceedings, lin2024promptda, title, author, journal, year, 2024, citation, generous, from, prof, experiments, space, unitree, also, jiahang, cao, xinyao, wentao, dong, their, setting, platform, collecting, zhengbang, zhu, weinan, zhang, acknowledgements, this, demo, implemented, vit, small, which, run, single, rtx, 4090, gpu, sped, demonstration, fps, policy, trained, only, diffuse, grasp, transparent, specular, ones, outperforming, replaced, vehicle, precision, scenes, testing, map, generated, api, using, 24x24, points, images, dense, preferred, its, noisy, due, generate, maps, after, aligning, struggle, consistent, scale, information, play, critical, role, unleashing, language, vision, specific, tasks, first, time, introduce, into, creating, paradigm, termed, specifically, guide, output, centers, concise, fusion, design, integrates, multiple, scales, within, decoder, address, training, challenges, posed, containing, both, precise, propose, includes, synthetic, simulation, real, pseudo, generation, sets, state, arts, arkitscenes, abstract, introduced, train, release, detailed, annotation, widely, available, taken, guiding, produce, inspired, success, vlm, llm, unleash, following, highlights, interactive, hugging, face, code, paper, eth, zurich, shanghai, jiao, tong, bytedance, seed, zhejiang,

Text of the page (random words):
prompt depth anything prompting depth anything for 4k resolution accurate metric depth estimation haotong lin 1 2 sida peng 1 jingxiao chen 3 songyou peng 4 jiaming sun 1 minghuan liu 3 hujun bao 1 jiashi feng 2 xiaowei zhou 1 bingyi kang 2 1 zhejiang university 2 bytedance seed 3 shanghai jiao tong university 4 eth zurich paper arxiv code hugging face interactive results dataset your browser does not support the video tag prompt depth anything is a high resolution and accurate metric depth estimation method with the following highlights we use prompting to unleash the power of depth foundation models inspired by success of prompting in vlm and llm foundation models the widely available iphone lidar is taken as the prompt guiding the model to produce up to 4k resolution accurate metric depth a scalable data pipeline is introduced to train our method we release a more detailed depth annotation for scannet dataset prompt depth anything benefits downstream applications including 3d reconstruction and generalized robotic grasping abstract prompts play a critical role in unleashing the power of language and vision foundation models for specific tasks for the first time we introduce prompting into depth foundation models creating a new paradigm for metric depth estimation termed prompt depth anything specifically we use a low cost lidar as the prompt to guide the depth anything model for accurate metric depth output achieving up to 4k resolution our approach centers on a concise prompt fusion design that integrates the lidar at multiple scales within the depth decoder to address training challenges posed by limited datasets containing both lidar depth and precise gt depth we propose a scalable data pipeline that includes synthetic data lidar simulation and real data pseudo gt depth generation our approach sets new state of the arts on the arkitscenes and scannet datasets and benefits downstream applications including 3d reconstruction and generalized robotic grasping comparisons with monocular depth methods monocular depth methods can generate high res depth maps but struggle with consistent metric scale information even after aligning with lidar comparisons with arkit lidar depth a dense lidar is accurate but high cost a low cost lidar is preferred but its depth is low res and noisy due to the limited power arkit lidar depth is a low resolution depth map generated by the arkit api using the 24x24 points iphone lidar and rgb images more testing results application street reconstruction the prompts can be replaced with vehicle lidar achieving high precision depth estimation in street scenes application generalized robotic grasping your browser does not support the video tag even if the grasping policy is trained only on diffuse objects our depth can help grasp transparent and specular ones outperforming rgb and lidar this demo is implemented with our vit small model which can be run in 94 fps on a single rtx 4090 gpu the video is sped up 2x for demonstration acknowledgements we thank the generous support from prof weinan zhang for robot experiments including the space objects and the unitree h1 robot we also thank zhengbang zhu jiahang cao xinyao li wentao dong for their help in setting up the robot platform and collecting robot data citation inproceedings lin2024promptda title prompting depth anything for 4k resolution accurate metric depth estimation author lin haotong and peng sida and chen jingxiao and peng songyou and sun jiaming and liu minghuan and bao hujun and feng jiashi and zhou xiaowei and kang bingyi journal arxiv year 2024 thanks to lior yariv for the website template thanks to nerfies for the useful add ons thanks to viser for the useful add ons