NCBI connection points as a reference (these are continually improved to provide good performance):

1.

NCBI Storage – High Performance Cluster Storage with multiple 10Gb connections to the NCBI backbone

2.

NCBI Network – 10gb connectivity or better to storage and key servers, and 20-40Gb to the core

3.

Upload/Download Server – 10gb card with multi-lane PCI and a current multi core processor. Optimized OS settings to support high bandwidth. For linux, see Red Hat tuning guide(1), for Windows the 2008 Tuning Guide(2)

4.

Server to Network – 10gb connection to switch

5.

NCBI Border – Current technology with 2 – 10gb connections

6.

Firewall – Redundant firewalls with 10gb connections

7.

Internet II connection – 10gb, Internet I –1gb(3)

8.

If you have an internet 1 connection, you will be limited in download performance as we currently have a 1 Gb connection – contact us to discuss Internet II options

Outside of NCBI:

1.

In order to support the increasing speeds for high performance data transfer, it is recommended to have a link to a research and education network rather than the commercial internet. In the US, that would be Internet II. Internationally, there are many links to the US via Internet II, each link generally supporting at least 2.5Gb/sec. You can check (map reference) for access to your country

2.

The connection from Internet II to your Organization should be 1Gb or better

3.

Network -> External Router – In general, from the entry point of your organization all the way through to your storage, you should support a minimum of 1Gb network connections.

4.

External Router – Must be able of supporting the data transfer speeds. It should be noted that a router that supports 10gb interfaces may only have 1-2Gb throughput. This is important and is worth checking.

5.

Router -> Firewall – must be at least as fast as external connection

6.

Firewall – Must be current technology, support UDP data transfer at external speeds (greater than 400Mb). This has often been a bottleneck at other sites. Just as with the router, firewalls that supports 10gb interfaces may only have 1-2Gb throughput. This is important and is worth checking (See Figure 2).

a.

Ensure the UDP ports 33001-33010 are permitted, that the timeout (if there exists one is longer than a data transfer, say 48 hours)

b.

If NATing, make sure the NAT timeout is longer than 48 hours.

c.

Ensure UDP throttling is not enabled. This will cause a connection to terminate after a set period of time or set amount of data transferred.

7.

Central Network -> Department – This network connection could be slow, sometimes 100Mb. It must be at least 1G, preferably 10Gb. Some organizations limit bandwidth at this connection, so even though the physical connection may be 1Gb, there may be policies that limit access. You should check with your organization’s network staff. On a campus, there may be value in a direct connection to the Internet II border firewall rather than upgrading the entire infrastructure on campus which might not require higher bandwidth.

8.

Departmental Firewall – This is often a slow point, one must ensure performance matches connection speeds. As above, be aware that some firewalls support a port speed higher than the actual processing ability. (example: Cisco ASA has 10Gb ports, but only supports 5Gb/sec with standard packet sizes)

9.

In building network – should be 1Gb or better connecting the firewall, servers and storage.

10.

Servers should be connected with 1Gb interfaces or better, and one need ensure there is no overallocation on the network switch. Further, even though monitoring of the network interface may show additional capacity exists, migrating to a higher performance 10gb connection has been shown to double performance on local networks.

a.

The network configuration defaults for linux need to be increased for high performance. Please see the redhat tuning, PSC tuning guide or DoE Fasternet.es.net/tuning.html.

Figure 2:

Notional Network Highlighting Common Performance Bottlenecks

# increase TCP max buffer size
net.core.rmem_max = 4194304
net.core.wmem_max = 4194304
# increase Linux autotuning TCP buffer limits
net.ipv4.tcp_rmem = 4096 87380 4194304 
net.ipv4.tcp_wmem = 4096 65536 4194304

11.

Storage subsystem – must be able to support 200MB/sec. with an internal RAID array or an external, high performance storage system (NAS). Note: Externally connected USB drives will often provide very poor performance and are not recommended. The storage subsystem is often the critical step in having a high performance download.

a.

On Linux, for local storage, alternative filesystems may provide better performance. Better performance has been seen with xfs, especially when there are large numbers of simultaneous writes and reads.