News Releases

iGrid V: Data and Lambda Services Demos

San Diego, CA, September 19, 2005 - This installment is the final describing iGrid demos to come next week! It focuses on data services and lambda services. It includes a project researching the origin of high-energy cosmic rays; a high-definition, underwater camera-based surveillance system focused on monitoring the sea coral near Taiwan; the transport of data over multi-Gigabit-per-second links without jitter; transport of extremely large files using Layer 1 of the network; transferring multi-gigabyte datasets to support earth science research; moving from Federal Express to lambda transport of large data over long distances; and dynamic user control of lightpaths to exchange large amounts of data among multiple locations.

The next installment will focus on the keynotes, panels, and master classes that will run in parallel to the demos at iGrid and point toward the “Future Generation Computer Systems (FGCS): The International Journal of Grid Computing: Theory, Methods and Applications,” a Special iGrid Issue in spring / summer 2006 that will capture the highlights of iGrid 2005.

Data Services

Transfer, Process, and Distribution of Mass Cosmic Ray Data fromTibet: The Yangbajing (YBJ) International Cosmic Ray Observatory is located in the YBJ valley of the Tibetan highland. The ARGO-YBJ Project is a Sino-Italian Cooperation, started in 2000 and fully operational in 2007, to research the origin of high-energy cosmic rays. It will generate more than 200 terabytes of raw data each year, which will then be transferred from Tibet to the Beijing Institute of High Energy Physics, where it will be processed and made available to physicists worldwide via a web portal. Chinese and Italian scientists are building a grid-based infrastructure to handle this data, which will have about 400 CPUs, mass storage, and broadband networking, a preview of which will be demonstrated at iGrid.

Data Reservoir on IPv6: 10-Gbps Disk Service in a Box: The Data Reservoir projects goal is to create a global grid infrastructure for scientific disciplines to enable distributed data sharing and high-speed computing for data analysis and numerical simulations. At the center of this infrastructure is the 2-Pflops system being developed as part of the GRAPE-DR project, to be operational in 2008. At iGrid, storage services that use the GRAPE-DRs single-box high-density 10-Gbps storage server and an IPv6 fast TCP data transfer protocol will demonstrate, for the first time, 10-Gbps TCP utilization in a production environment. By using inter-layer coordination optimizationfor TCP on both IPv4 and IPv6, 10-Gbps storage services can be realized using only a few TCP streams, providing new opportunities for large-scale data sharing. This project receives funding from the Special Coordination Fund for Promoting Science and Technology, MEXT, Japan.

Real-Time, Observational, Multiple Data Streaming and Machine Learning for Environmental Research Using Lightpaths: The Ecogrid is a grid-based ecological and biological surveillance system that uses sensors and video cameras to monitor Taiwans coral reef ecosystem and wildlife behavior. Based on grid technologies, the Ecogrid can be regarded as an open-layered pipeline: The sensors collect raw images and transmit information over IP; grid middleware is used for distributed computing and data storage; and the data is analyzed, synthesized, and archived for scientific discovery. At iGrid, images from high-definition underwater monitoring cameras in the coral reef reserve of Kenting, Taiwan, and from an innovative omni-directional camera from Osaka, will be streamed to San Diego, demonstrating the technologies that have been developed for real-time data streaming. Also to be shown are image streaming, being studied as a way to do machine learning. Using UCLP, a single end user controls the setup of lightpaths between Taiwan and North America.

TWAREN (Taiwan Advanced Research and Educational Network), provided by NCHC, provides 2.5 Gbps between Taiwan and the U.S. This application is part of the iGrid demo Worlds First Demonstration of X GRID Application Switching Using User Controlled LightPaths.

Exploring Remote and Distributed Data Using Teraflows: High-performance data services for data exploration, data integration, and data analysis are demonstrated; these services are designed to scale to 1-Gbps and 10-Gbps networks. These services are layered over the High-Performance Protocols (HPP) toolkit. In turn, these network protocols are layered over services for setting up, monitoring, and tearing down optical paths. Applications built with these data services demonstrate what are sometimes called photonic data services,i.e., advanced Layer-3 protocols and services closely integrated with Layer-2 and Layer-1 services. The latter is based on dynamic lightpath switching and supported by advanced optical control planes and leading-edge photonic technologies.

LightForce: High-Performance Data Multicast Enabled by Dynamic Lightpaths: Advanced applications require reliable point-to-point transport of extremely large data files among multiple nodes. Currently no communication service meets these data transport needs cost effectively. LightForce demonstrates the potential for transporting multiple gigabits of data with virtually no jitter by exploiting the capabilities of dynamic lightpath switching. It uses dynamic Layer-1 services and rapidly changing topologies that integrate multiple source and destination nodes, including remote nodes. Key technologies demonstrated include transparent mapping, lightpath control algorithms, resource allocation, arbitration among contending demands, extended FEC, error control, traffic stream quality, performance monitoring, and new protocols that exploit enhanced methods for low levels of BER.

DataWave: Ultra-High-Performance File Transfer Enabled By Dynamic Lightpaths: Science applications generate extremely large files that must be globally distributed. Most network performance methods focus on memory-to-memory data transfers as opposed to data-file-to-data-file. The DataWave project demonstrates the potential for transferring extremely large files internationally using dynamic Layer 1 services.

Real-Time Multi-Scale Brain Data Acquisition, Assembly, and Analysis Using an End-to-End OptIPuter: The operation of a scientific experiment on a global testbed consisting of visualization, storage, computational, and network resources will be demonstrated. These resources are bundled into a unified platform using OptIPuter-developed technologies, including dynamic lambda allocation, advanced transport protocols, the Distributed Virtual Computer (DVC) middleware, and a multi-resolution visualization system running over the Scalable Adaptive Graphics Environment (SAGE). Over these layered technologies, a multi-scale correlated microscopy experiment can be run in which a biologist images a sample and progressively magnifies it, zooming from an entire system, such as a rat cerebellum, to an individual spiny dendrite. Using both the 100-megapixel LambdaVision display and the auto-stereo Personal Varrier, scientists can effectively and simultaneously view every step of a multi-scale microscopy correlation process, viewing large 2-D scenes and 3-D subsections of the scene while comparing them to dozens of possible contexts and matching these to live video output of an electron microscope.

Scientific Collaboration with Parallel, Interactive, 3-D Visualizations of Earth Science Datasets: The ability to transfer multi-gigabyte objects among remote collaborating sites is an enabling technology for the Earth sciences. For this demonstration, the OptIPuters Distributed Virtual Computer (DVC) middleware is used to establish a collaborative environment with visualization endpoints at UCSD and UCI. OptIPuters Composite Endpoint Protocol (CEP) and Group Transport Protocol (GTP), two cluster-to-cluster transport protocols, are used to efficiently send multi-gigabyte, 3-D scene files from the various storage sites for interactive visualization on tiled displays at the visualization sites.

From Federal Express to Lambdas: Transporting SDSS Data Using UDT: Until now, Sloan Digital Sky Survey (SDSS) datasets have been shared with collaborators using Federal Express. Optical paths and new transport protocols are enabling these datasets to be transported using networks over long distances. The SDSS DR3 multi-terabyte dataset will be transported among Chicago and various sites in Europe and Asia using NCDMs data transport protocol UDT (UDP-based Data Transport Protocol) over the Teraflow Testbed. The Teraflow Testbed is an infrastructure designed to test new 10-Gbps network protocols and data services with high-volume data flows, or teraflows, over both traditional routed networks as well as optical networks.

Lambda Services

Worlds First Demonstration of X GRID Application Switching Using User-Controlled LightPaths: Researchers from CANARIE and CRC in Canada, NCHC in Taiwan, KISTI in Korea, and i2CAT in Spain are collaborating on the worlds first demonstration of multiple end users dynamically controlling the setup and switching of lightpaths to various grid application resources worldwide using CANARIEs UCLP (User Controlled LightPath) management software. In the future, grid users will be able to quickly configure (and reconfigure) global optical lightpaths among collaborating research institutions so their applications can exchange data in real time. While other iGrid 2005 demos may also use UCLP, these groups are accessing resources at each others institutions to demonstrate how one controls 40,000 kilometer of lightpaths. Applications driving these demonstrations are include the following:

Coordination of Grid Scheduler and Lambda Path Service over GMPLS: Toward Commercial Lambda Path Service: In the future, grid applications developers will be able to schedule lightpaths provided by commercial network providers. In this demonstration, a large-scale nano-quantum application will utilize both computational resources and an optical path. The application uses Ninf-G middleware, which calls on a Scheduler that calls on a Lambda Path Resource Manager to request an optical path in a GMPLS control environment, specifying the end nodeslocations and other requirements, such as bandwidth. GMPLS, in turn, programs physical OXCs (optical cross connects) to set up end-to-end lightpaths over multiple domains. The goal is to define a standard interface between the grid and the optical network, which can be used to realize emerging new commercial services that both applications service providers and commercial network companies will accept.

First Functional Demonstration of OVC / Terabit LAN with SAGE: The Terabit LAN / WAN project focuses on parallelism in switching and transporting multiple lambdas. Switching is done by the applications; applications create a Lambda Group Path (LGP) and specify the number of lambdas, or channels, in each LGP. Each LGP is treated logically as one end-to-end optical path, so during parallel transport, the LGP channels have identical optical paths, and no relative latency deviation occurs. However, optical path diversity (due to restoration) or group velocity dispersion in optical fibers can cause LGP relative latency deviations and negatively affect quality of service, particularly in applications involving streaming media.

NTTs OVC (Optical Virtual Concatenation) hardware compensates for relative latency deviations to achieve a virtual terabit bulk transport. Using the EVL-developed SAGE application, 2x1GE video streams will be sent from Chicago to San Diego for display on a 2x1 tiled display. An LGP with two channels is established. Tests are performed in which the channels have identical paths and then different paths, and with OVC turned onand off.OVC eliminates relative latency deviation, or jitter, which is visually apparent as poor synchronization between the two images on the tiles of the display. OVC is realized by the OTN (Optical Transport Network) function defined in ITU-T recommendation G.709.

Dynamic Provisioning Across Administrative Domains: Dynamic Resource Allocation via GMPLS Optical Networks (DRAGON) demonstrates dynamic provisioning across multiple administrative domains to enable Very Long Baseline Interferometry (VLBI). Using simultaneous observations from radio telescopes in the USA (MIT Haystack), Japan (Kashima) and Europe (Onsala in Sweden, Jodrell in the UK, Westerbork in The Netherlands), scientists collect data to create ultra-high-resolution images of distant objects and make precise measurements of the Earths motion in space. The HOPI testbed facilitates connectivity between various domains across the globe to simultaneously transfer VLBI data from these stations to the MIT Haystack Observatory at 512-Mb/station where the data will be cross-correlated in real time, then sent to iGrid where the results will be visualized in real time.

Global Lambdas for Particle Physics Analysis: Caltech, SLAC, and FNAL use advanced networks to demonstrate analysis tools that will enable physicists to control worldwide grid resources when analyzing major high-energy physics events. Components of this Grid Analysis Environmentare being developed by such projects as UltraLight, FAST, PPDG, GriPhyN, and iVDGL. The iGrid demonstration will show how a physicist can issue on-demand network and resource provisioning in response to event analysis requests from his desktop computer. A requests complex workflows are translated using provisioning algorithms into network flow allocations and scheduled resource bookings on remote computers / clusters. Caltechs MonALISA monitoring framework illustrates the progress of the analysis tasks, data flows in the network, and the effects on the global system.

International 10-Gbps Line-Speed Security: This security demonstration will use AES-256 encryption and OME switching hardware to send encrypted streaming media at 10-Gbps speeds with minimal latency from Nortels Ottawa facility and Amsterdam through StarLight in Chicago to iGrid in San Diego. Such capability is desirable for many applications of lightpaths to reduce vulnerability to theft or modification of commercially valuable information. Applications can enable and disable encryption on demand during iGrid.

IPv4 Link-Local IP Addressing for Optical Networks: When setting up lightpaths, the end nodes use IP addresses to communicate. Address assignment is now done manually. An automatic solution will be demonstrated that uses IPv4 link-local addresses while retaining the authenticity of the hosts.

SPIN and ISON to Support Emerging Collaborative Applications: As deployment of local LambdaGrid domains based on wavelength-routing optical networks gains momentum, there is increasing demand for interoperation among them to enable deployment of global collaborative grid applications. The SPIN (Secure Photonic Interdomain Negotiator) network control plane enables seamless deployment of global collaborative grid applications over a multi-domain, wide-area LambdaGrid with secure inter-domain access to shared optical resources. In addition, an increasing number of collaborative grid applications require multimedia traffic support as they generate variable-rate streaming and bursty traffic. The Integrated Services Optical Network (ISON) enables a multi-purpose LambdaGrid capable of efficiently supporting multimedia collaborative grid applications with diverse networking bandwidth and communication requirements. At iGrid, scientists will use a SPIN-based optical networking monitor and an ISON-based dataset portal application to post, mine, and visualize scientific datasets in a multi-domain LambdaGrid.

Token-Based Network Element Access Control and Path Selection: For grid computing to successfully interface with control planes and firewalls, new security techniques must be developed. Traditional network access security models use an outsourcingmodel or an (OGSA-based) configurationmodel. The push,or token, model, which will be demonstrated at iGrid, works at lower network levels. In this model, an applications or users access rights are determined by a token issued by an authority. The token is used to signal the opening of the data path. The advantage of using tokens is that a path can be pre-provisioned and an application or user holding tokens can access the network resource potentially faster then in the other models.

VM Turntable: Making Large-Scale Remote Execution More Efficient and Secure with Virtual Machines Riding on Dynamic Lightpaths: Traditional grid computing focuses on remote computing and data analysis, while the underlying network resources are treated as separate and inflexible. Recent advances in controllable and dynamic lightpaths extend the meaning of remote executionto include the networks themselves, creating an environment that permits orchestration of computation+data+network resources. With this capability, three problems associated with remote execution are addressed: (a) the elusive locality of data references (with computation and working datasets that often end up being separated by a whole ocean), (b) confidentiality and integrity (with sensitive programs and/or data at the mercy of subtly compromised, bugged hosting environments), and (c) portability, versioning woes in the presence of end systemscomplex software stacks. The VM Turntable is structured around Xen-based Linux Virtual Machines that can be migrated in real time while supporting live applications transporting the whole set of memory pages and hard disk contents to various destinations. The live migration of Virtual Machines exploits a high degree of pipelining between the staggered operations of assembling the data to be transferred, verifying its integrity, and finally halting and transferring execution. To maintain lightpath security, the VM Turntable utilizes a token-based approach to efficiently enforce policies at the bearer and control path levels (see the iGrid application Token-based Network Element Access Control and Path Selection). At iGrid 2005, the VM Turntable will live-migrate the execution of a search-refine iterative workflow against unique datasets located in Amsterdam, Chicago, and San Diego.

Related Links
www.calit2.net/articles/article.php?id=690