Meta tags:
description= Using the right tools, you should be able to move 1 TB between sites in less than 8 hours.;
Headings (most frequently used words):
background, selecting, file, transfer, tool,
Text of the page (most frequently used words):
the (14), tools (13), data (12), and (10), #transfer (10), globus (7), performance (6), background (6), you (5), require (5), file (5), files (5), issues (5), server (4), http (4), can (4), are (4), encrypted (4), tuning (4), other (3), ftp (3), install (3), tool (3), software (3), science (3), option (3), scp (3), sftp (3), rsync (3), ssh (3), transfers (3), parallel (3), number (3), nodes (3), network (3), dtn (3), recommend (2), not (2), have (2), set (2), based (2), system (2), administrator (2), web (2), only (2), everything (2), user (2), that (2), long (2), fdt (2), general (2), most (2), over (2), hpnssh (2), control (2), but (2), selecting (2), security (2), into (2), smaller (2), include (2), large (2), copy (2), more (2), larger (2), using (2), for (2), doe (2), site (2), quic (2), protocol (2), tpc (2), cloud (2), storage (2), commercial (2), command (2), line (2), hpn (2), say (2), home (2), nsf (2), esnet (2), host (2), testing (2), hardware (2), selection (2), dmz (2), buffer (2), last, edited, 2025, this, time, having, best, combination, features, support, issue, whether, requirement, ability, such, sshd, installed, else, normal, does, any, endpoints, source, destination, locations, known, within, open, projects, seem, prefer, wan, possibly, depending, your, throughput, requirements, disk, https, both, channels, password, channel, unencrypted, anonymous, anyone, access, when, one, first, things, decide, which, model, basic, options, common, technique, speed, break, pieces, transferred, copying, several, once, typically, good, try, efficient, than, bundling, multiple, small, single, tar, zip, also, recommended, historically, wide, area, bulk, has, been, plagued, poor, variety, reasons, these, improper, configuration, sending, receiving, hosts, design, firewalls, factors, cases, however, sets, moved, distances, today, networks, with, minimal, effort, campus, plans, iren, docs, dtns, perfsonar, troubleshooting, test, expectations, 100g, scorecard, contributed, configurations, reference, implementation, 2021, version, community, presentation, learn, faq, practices, policy, monitoring, components, architecture, motivation, bloat, mtu, firewall, router, tcp, explained, containers, windows, freebsd, mac, osx, linux, information, page, myesnet, menu,
Text of the page (random words):
background menu myesnet esnet home page host tuning background information linux mac osx freebsd ms windows containers network tuning tcp issues explained buffer issues router tuning firewall issues mtu issues buffer bloat science dmz motivation architecture network components performance monitoring security practices and policy faq learn more science dmz community presentation data transfer nodes hardware selection hardware selection 2021 version reference implementation contributed configurations dtn tuning dtn software dtn performance testing data transfer scorecard data transfer tools background say no to scp sftp rsync globus hpn ssh command line tools commercial tools cloud storage tools http tpc quic protocol doe site data transfer nodes performance testing performance expectations for a 100g host network test tools troubleshooting perfsonar esnet dtns nsf docs nsf iren campus ci plans home data transfer tools background data transfer tools background say no to scp sftp rsync globus hpn ssh command line tools commercial tools cloud storage tools http tpc quic protocol doe site data transfer nodes background historically wide area bulk data transfer has been plagued by poor performance for a variety of reasons these include improper configuration of the sending and receiving hosts software design issues firewalls and other factors in most cases however large data sets can be moved long distances using today s networks with minimal effort a common technique to speed up file transfers is to break the file into smaller pieces that are transferred in parallel a number of tools include the option to do parallel transfers if you have a large number of files to copy you can do parallel transfers by copying several files at once typically 4 5 is a good number to try but in general it is more efficient to copy larger files than smaller files so bundling multiple small files into a single larger file using tar or zip is also recommended selecting a file transfer tool when selecting a file transfer tool one of the first things to decide is which security model you require the basic set of options are anonymous e g ftp http anyone can access the data password encrypted e g globus fdt control channel is encrypted but data is unencrypted everything encrypted e g scp sftp rsync over ssh hpnssh globus https based web server both control and data channels are encrypted in general most open science projects seem to prefer option 2 if you require option 3 over a wan we recommend globus or possibly hpnssh depending on your throughput requirements and disk performance the other issue is whether or not you have the requirement and or ability to set up a server http and ftp based tools require a system administrator to install a web or ftp server other tools such as fdt only require an sshd server to be installed by the administrator and everything else can be install as a normal user globus is the only tool that does not require the user to install any software as long as the endpoints source and destination locations are known within the system at this time we recommend globus as having the best combination of features and support last edited 01 05 2025
|
