John H Hartman

Interests

Research

Cloud computing, distributed computing, storage systems, operating systems, and computer networks.

Courses

Scholarly Contributions

Journals/Publications

• Choi, I., Ponsero, A. J., Bomhoff, M., Youens-Clark, K., Hartman, J. H., & Hurwitz, B. L. (2019). Libra: scalable k-mer--based tool for massive all-vs-all metagenome comparisons. GigaScience, 8(2), giy165.
• Youens-Clark, K., Bomhoff, M., Ponsero, A. J., Wood-Charlson, E. M., Lynch, J., Choi, I., Hartman, J. H., & Hurwitz, B. L. (2019). iMicrobe: Tools and data-driven discovery platform for the microbiome sciences. GigaScience, 8(7).
• Choi, I., Ponsero, A. J., Youens-Clark, K., Bomhoff, M., Hartman, J. H., & Hurwitz, B. L. (2018). Libra: scalable k-mer based tool for massive all-vs-all metagenome comparisons. GigaScience.
• Jha, A. K., Kupinski, M. A., Barrett, H. H., Clarkson, E., & Hartman, J. H. (2012). Three-dimensional Neumann-series approach to model light transport in nonuniform media. Journal of the Optical Society of America A: Optics and Image Science, and Vision, 29(9), 1885-1899.
  More info

  PMID: 23201945;PMCID: PMC3963433;Abstract: We present the implementation, validation, and performance of a three-dimensional (3D) Neumann-series approach to model photon propagation in nonuniform media using the radiative transport equation (RTE). The RTE is implemented for nonuniform scattering media in a spherical harmonic basis for a diffuse-optical-imaging setup. The method is parallelizable and implemented on a computing system consisting of NVIDIA Tesla C2050 graphics processing units (GPUs). The GPU implementation provides a speedup of up to two orders of magnitude over non-GPU implementation, which leads to good computational efficiency for the Neumann-series method. The results using the method are compared with the results obtained using the Monte Carlo simulations for various small-geometry phantoms, and good agreement is observed. We observe that the Neumann-series approach gives accurate results in many cases where the diffusion approximation is not accurate. © 2012 Optical Society of America.
• Lei, Y. e., Gniady, C., & Hartman, J. H. (2011). Energy-efficient memory management in virtual machine environments. 2011 International Green Computing Conference and Workshops, IGCC 2011.
  More info

  Abstract: Main memory is one of the primary shared resources in a virtualized environment. Current trends in supporting a large number of virtual machines increase the demand for physical memory, making energy efficient memory management more significant. Several optimizations for memory energy consumption have been recently proposed for standalone operating system environments. However, these approaches cannot be directly used in a virtual machine environment because a layer of virtualization separates hardware from the operating system and the applications executing inside a virtual machine. We first adapt existing mechanisms to run at the VMM layer, offering transparent energy optimizations to the operating systems running inside the virtual machines. Static approaches have several weaknesses and we propose a dynamic approach that is able to optimize energy consumption for currently executing virtual machines and adapt to changing virtual machine behaviors. Through detailed trace driven simulation, we show that proposed dynamic mechanisms can reduce memory energy consumption by 63.4% with only 0.6% increase in execution time as compared to a standard virtual machine environment. © 2011 IEEE.
• Lei, Y. e., Gen, L. u., Kumar, S., Gniady, C., & Hartman, J. H. (2010). Energy-efficient storage in virtual machine environments. VEE 2010 - Proceedings of the 2010 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, 75-84.
  More info

  Abstract: Current trends in increasing storage capacity and virtualization of resources combined with the need for energy efficiency put a challenging task in front of system designers. Previous studies have suggested many approaches to reduce hard disk energy dissipation in native OS environments; however, those mechanisms do not perform well in virtual machine environments because a virtual machine (VM) and the virtual machine monitor (VMM) that runs it have different semantic contexts. This paper explores the disk I/O activities between VMM and VMs using trace driven simulation to understand the I/O behavior of the VM system. Subsequently, this paper proposes three mechanisms to address the isolation between VMM and VMs, and increase the burstiness of hard disk accesses to increase energy efficiency of a hard disk. Compared to standard shutdown mechanisms, with eight VMs the proposed mechanisms reduce disk spin-ups, increase the disk sleep time, and reduce energy consumption by 14.8% with only 0.5% increase in execution time. We implemented the proposed mechanisms in Xen and validated our simulation results. Copyright © 2010 ACM.
• Perianayagam, S., Andrews, G. R., & Hartman, J. H. (2010). Rex: A toolset for reproducing software experiments. Proceedings - 2010 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2010, 613-617.
  More info

  Abstract: Software experiments such as BLAST are hard to reproduce even if identical hardware is available because external data sets could have changed, software used in the original experiment may be unavailable, or the input parameters for the experiment may not be documented. This paper presents Rex, a toolset that allows one to record an experiment and archive its apparatus, replay an experiment, conduct new experiments on a recorded apparatus, and compare differences between experiments. The execution time overhead of recording experiments is on average about 2%; the space overhead of an archive ranges from 8 MB to 5 GB. ©2010 IEEE.
• Crk, I., Albinali, F., Gniady, C., & Hartman, J. (2009). Understanding energy consumption of sensor enabled applications on mobile phones. Proceedings of the 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society: Engineering the Future of Biomedicine, EMBC 2009, 6885-6888.
  More info

  Abstract: Recent research in ubiquitous and mobile computing uses mobile phones and wearable accelerometers to monitor individuals' physical activities for personalized and proactive health care. The goal of this project is to measure and reduce the energy demand placed on mobile phones that monitor individuals' physical activities for extended periods of time with limited access to battery recharging and mobile phone reception. Many issues must be addressed before mobile phones become a viable platform for remote health monitoring, including: security, reliability, privacy, and, most importantly, energy. Mobile phones are battery-operated, making energy a critical resource that must be carefully managed to ensure the longest running time before the battery is depleted. In a sense, all other issues are secondary, since the mobile phone will simply not function without energy. In this project, we therefore focus on understanding the energy consumption of a mobile phone that runs MIT wockets, physical activity monitoring applications, and consider ways to reduce its energy consumption.
• Cappos, J., Samuel, J., Baker, S., & Hartman, J. H. (2008). A look in the mirror: Attacks on package managers. Proceedings of the ACM Conference on Computer and Communications Security, 565-574.
  More info

  Abstract: This work studies time security of ten popular package managers. These package managers use different security mechanisms that provide varying levels of usability and resilience to attack. We find that, despite their existing security mnechanisms, all of these package managers have vulnerabilities that can be exploited by a man-in-the-middle or a malicious mirror. While all current package managers suffer frons vulnerabilities. their security is also positively or negatively impacted by the distribution's security practices. Weaknesses in package managers are more easily exploited when distributions use third-party nurrors as official mirrors. We were successful in using false credentials to obtain an official nurror on all five of the distributions we attempted. We also found that some security mechanisms that control where a client obtains metadata and packages frotn may actually decrease security. We analyze current package managers to show that by exploiting vulnerabilities, an attacker with a mirror can compromise or crash hundreds to thousands of clients weekly. The problemns we disclose are now being corrected by many different package manager maintainers. Copyright 2008 ACM.
• Huang, H., Hartman, J. H., & Hurst, T. N. (2007). Data-centric routing in sensor networks using biased walk. 2006 3rd Annual IEEE Communications Society on Sensor and Adhoc Communications and Networks, Secon 2006, 1, 1-9.
  More info

  Abstract: We present Spiral, a data-centric routing algorithm for short-term communication in unstructured sensor networks. Conventional data-centric routing algorithms are based on flooding or random walk. Flooding returns the shortest route but has a high search cost; random walk has a lower search cost but returns a sub-optimal route. Spiral offers a compromise between these two extremes -it has a lower search cost than flooding and returns better routes than random walk. Spiral is a biased walk that visits nodes near the source before more distant nodes. This results in a spiral-like search path that is not only more likely to And a closer copy of the desired data than random walk, but is also able to compute a shorter route because the network around the source is more thoroughly explored. Our experiments show that in a 500-node network with an average degree of 20 and two copies of every data object, for a short-term communication of 40 packets the total communication cost by Spiral is only 72% of that by flooding, 81% of ERS, 74% of random walk, and 73% of DFS. © 2006 IEEE.
• Huang, H., Hartman, J. H., & Hurst, T. N. (2007). Efficient and robust query processing for mobile wireless sensor networks. International Journal of Sensor Networks, 2(1-2), 99-107.
  More info

  Abstract: We present CNFS, an algorithm for efficient and robust query processing for mobile wireless sensor networks. CNFS is a walk-based algorithm that is biased to visit nodes close to the source first. This bias is accomplished by collecting topology information about the network as the search progresses. This information is also used to compute the shortest return path for the query result and to tolerate changes in the network topology caused by node mobility that could otherwise cause the query to fail. As a result, CNFS requires fewer messages to process a query than flooding-based algorithms, while tolerating node mobility better than random walk-based algorithms. Our experiments show that in medium-density networks (average node degree 8.3) CNFS requires about 38% fewer messages than the other algorithms studied, while experiencing significantly fewer query failures than random walk-based algorithms in both sparse and dense networks. CNFS's success rate is comparable to flooding-based algorithms in dense networks and slightly worse in sparse networks. Copyright © 2007 Inderscience Enterprises Ltd.
• Hartman, J. H., Baker, S., & Murdock, I. (2006). Customizing the swarm storage system using agents. Software - Practice and Experience, 36(2), 117-137.
  More info

  Abstract: Swarm is a scalable, modular storage system that uses agents to customize low-level storage functions to meet the needs of high-level services. Agents influence low-level storage functions such as data layout, metadata management, and crash recovery. An agent is a program that is attached to data in the storage system and invoked when events occur during the data's lifetime. For example, before Swarm writes data to disk, agents attached to the data are invoked to determine a layout policy. Agents are typically persistent, remaining attached to the data they manage until the data are deleted; this allows agents to continue to affect how the data are handled long after the application or storage service that created the data has terminated. In this paper, we present Swarm's agent architecture, describe the types of agents that Swarm supports and the infrastructure used to support them, and discuss their performance overhead and security implications. We describe how several storage services and applications use agents, and the benefits they derive from doing so. Copyright © 2005 John Wiley & Sons, Ltd.
• Huang, H., Hartman, J. H., & Hurst, T. N. (2006). Efficient and robust query processing for mobile wireless sensor networks. GLOBECOM - IEEE Global Telecommunications Conference.
  More info

  Abstract: We present CNFS, an algorithm for efficient and robust query processing for mobile wireless sensor networks. CNFS is a walk-based algorithm that is biased to visit nodes close to the source first. This bias is accomplished by collecting topology information about the network as the search progresses. This information is also used to tolerate changes in the network topology caused by node mobility that could otherwise cause the query to fail. As a result, CNFS requires fewer messages to process a query than flooding-based algorithms, while tolerating node mobility better than random walk-based algorithms. Our experiments show that in medium-density networks (average node degree 8.3) CNFS requires about 37% fewer messages than the other algorithms studied, while experiencing significantly fewer query failures than random walk-based algorithms in both sparse and dense networks. CNFS's success rate is comparable to flooding-based algorithms in dense networks and slightly worse in sparse networks. © 2006 IEEE.
• Muir, S., Peterson, L., Fiuczynski, M., Cappos, J., & Hartman, J. (2006). Privileged operations in the PlanetLab virtualised environment. Operating Systems Review (ACM), 40(1), 75-88.
  More info

  Abstract: Virtualised systems have experienced a resurgence in popularity in recent years, whether used to support multiple OSes running on a user's desktop, provide commercial application hosting facilities, or isolate a large number of users from each other in global network testbeds. We also see an increasing level of interest in having entities within these virtualised systems interact with each other, either as peers or as helpers providing a service to clients. Very little work has been previously conducted on how such interaction between virtualised environments can take place. We introduce Proper, a service running on the PlanetLab system, that allows unprivileged entities to access privileged operations in a safe, tightly controlled manner. This paper describes our work designing and implementing Proper, including a discussion of the various architectural decisions made. We describe how implementing such a system in a traditional UNIX environment is non-trivial, and provide a number of examples of how services running on PlanetLab actually use Proper.
• Baker, S., & Hartman, J. H. (2004). The mirage NFS router. Proceedings - Conference on Local Computer Networks, LCN, 242-249.
  More info

  Abstract: Mirage 1 aggregates multiple NFS servers into a single, virtual NFS file server. It is interposed between the NFS clients and servers, making the clients believe that they are communicating with a single, large server. Mirage is an NFS router because it routes an NFS request from a client to the proper NFS server, and routes the reply back to the proper client. Experiments with a Mirage prototype show that Mirage effectively virtualizes an NFS server using unmodified clients and servers. Mirage imposes a negligible overhead on a realistic NFS workload. On real world workloads, such as a collection of clients executing compile jobs over NFS, Mirage imposes an overhead of 3% as compared to a proxy that simply forwards packets. © 2004 IEEE.
• Baker, S., Hartman, J. H., & Murdock, I. (2004). Swarm: Agent-based storage. Proceedings of the International Conference on Software Engineering Research and Practice, SERP'04, 2, 483-489.
  More info

  Abstract: Swarm is a scalable, modular storage system that uses agents to customize low-level storage functions to meet the needs of high-level services. Agents influence low-level storage functions such as data layout, metadata management, and crash recovery. An agent is a program that is attached to data in the storage system and invoked when particular events occur during the data's lifetime. For example, when Swarm needs to write data to disk, agents attached to the data are invoked to determine a layout policy. Agents can be persistent, so that they remain attached to the data they manage until the data are deleted; this allows agents to continue to affect how the data are handled long after the application or storage service that created the data has terminated. In this paper, we present Swarm's agent architecture, describe the types of agents that Swarm supports and the infrastructure used to support them, and discuss their performance overhead and security implications. We describe how several storage services and applications use agents, and the benefits they derive from doing so.
• Baker, S., & Hartman, J. H. (2001). The design and implementation of the Gecko NFS Web proxy. Software - Practice and Experience, 31(7), 637-665.
  More info

  Abstract: The World Wide Web uses a naming scheme (URLs), transfer protocol (HTTP), and cache algorithms that are different from traditional network file systems. This makes it impossible for unmodified Unix applications to access the Web; as a result, the Web is only accessible to special Web-enabled applications. Gecko bridges the gap between Web-enabled applications and conventional applications, allowing any unmodified Unix application to access the Web. We developed two prototypes of the Gecko system at the University of Arizona and incorporated the many lessons learned from the first into the second. Minimizing the amount of state was a key goal of the Gecko redesign and the second prototype uses a unique compression technique to reduce server state. Experiments performed on the prototype show that Gecko not only makes the Web accessible to unmodified applications, but does so at a performance that meets or exceeds that of HTTP. © 2001 John Wiley & Sons, Ltd.
• Peterson, L., Gottlieb, Y., Hibler, M., Tullmann, P., Lepreau, J., Schwab, S., Dandekar, H., Purtell, A., & Hartman, J. (2001). An OS interface for active routers. IEEE Journal on Selected Areas in Communications, 19(3), 473-487.
  More info

  Abstract: This paper describes an operating system (OS) interface for active routers. This interface allows code loaded into active routers to access the router's memory, communication, and computational resources on behalf of different packet flows. In addition to motivating and describing the interface, the paper also reports our experiences implementing the interface in three different OS environments: Scout, the OSKit, and the exokernel.
• Sarkar, P., & Hartman, J. H. (2000). Hint-based cooperative caching. ACM Transactions on Computer Systems, 18(4), 387-419.
  More info

  Abstract: This article presents the design, implementation, and measurement of a hint-based cooperative caching file system. Hints allow clients to make decisions based on local state, enabling a loosely coordinated system that is simple to implement. The resulting performance is comparable to that of existing tightly coordinated algorithms that use global state, but with less overhead. Simulations show that the block access times of our system are as good as those of the existing algorithms, while reducing manager load by more than a factor of seven, block lookup traffic by nearly a factor of two-thirds, and replacement traffic a factor of five. To verify our simulation results in a real system with real users, we implemented a prototype and measured its performance for one week. Although the simulation and prototype environments were very different, the prototype system mirrored the simulation results by exhibiting reduced overhead and high hint accuracy. Furthermore, hint-based cooperative caching reduced the average block access time to almost half that of NFS.
• Spalink, T., Hartman, J. H., & Gibson, G. A. (2000). Mobile agent's effects on file service. IEEE Concurrency, 8(2), 62-69.
  More info

  Abstract: Implementing an application using mobile agents might or might not improve its performance. The effects of moving an agent application from a client to a file server are analyzed. The circumstances under which application performance is improved are studied. It is shown if these circumstances come at the expense of other applications using the same server.
• Spatscheck, O., Hansen, J. S., Hartman, J. H., & Peterson, L. L. (2000). Optimizing TCP forwarder performance. IEEE/ACM Transactions on Networking, 8(2), 146-157.
  More info

  Abstract: A TCP forwarder is a network node that establishes and forwards data between a pair of TCP connections. An example of a TCP forwarder is a firewall that places a proxy between a TCP connection to an external host and a TCP connection to an internal host, controlling access to a resource on the internal host. Once the proxy approves the access, it simply forwards data from one connection to the other. We use the term TCP forwarding to describe indirect TCP communication via a proxy in general. This paper briefly characterizes the behavior of TCP forwarding, and illustrates the role TCP forwarding plays in common network services like firewalls and HTTP proxies. We then introduce an optimization technique, called connection splicing, that can be applied to a TCP forwarder, and report the results of a performance study designed to evaluate its impact. Connection splicing improves TCP forwarding performance by a factor of two to four, making it competitive with IP router performance on the same hardware. © 2000 IEEE.
• Baker, S., & Hartman, J. H. (1999). Gecko NFS Web proxy. Computer Networks, 31(11), 1725-1736.
  More info

  Abstract: The World-Wide Web provides remote access to pages using its own naming scheme (URLs), transfer protocol (HTTP), and cache algorithms. Not only does using these special-purpose mechanisms have performance implications, but they make it impossible for standard Unix applications to access the Web. Gecko is a system that provides access to the Web via the NFS protocol. URLs are mapped to Unix file names, providing unmodified applications access to Web pages; pages are transferred from the Gecko server to the clients using NFS instead of HTTP, significantly improving performance; and NFS's cache consistency mechanism ensures that all clients have the same version of a page. Applications access pages as they would Unix files. A client-side proxy translates HTTP requests into file accesses, allowing existing Web applications to use Gecko. Experiments performed on our prototype show that Gecko is able to provide this additional functionality at a performance level that exceeds that of HTTP.
• Hartman, J. H., Murdock, I., & Spalink, T. (1999). Swarm scalable storage system. Proceedings - International Conference on Distributed Computing Systems, 74-81.
  More info

  Abstract: Swarm is a storage system that provides scalable, reliable, and cost-effective data storage. Swarm is based on storage servers, rather than file servers; the storage servers are optimized for cost-performance and aggregated to provide high-performance data access. Swarm uses a striped log abstraction to store data on the storage servers. This abstraction simplifies storage allocation, improves file access performance, balances server loads, provides fault-tolerance through computed redundancy, and simplifies crash recovery. We have developed a Swarm prototype using a cluster of Linux-based personal computers as the storage servers and clients; the clients access the servers via the Swarm-based Sting file system. Our performance measurements show that a single Swarm client can write to two storage servers at 3.0 MB/s., while four clients can write to eight servers at 16.0 MB/s.
• Hartman, J. H. (1997). Index-based hyperlinks. Computer Networks, 29(8-13), 1129-1135.
  More info

  Abstract: We propose a new mechanism for implicitly specifying hyperlinks in HTML documents using indices. Indices are dictionaries that associate keys (words or phrases) with one or more attributes. Indices maintain these key/attribute bindings over all or part of a document, and are used by browsers to create hyperlinks dynamically. Indices may also include bindings of other indices, in a hierarchical fashion. We argue that indices are both simpler and more general than the current HTML hyperlink mechanism. We have developed a prototype browser that uses index-based hyperlinks. © 1997 Published by Elsevier Science B.V.
• Hartman, J. H., & Ousterhout, J. K. (1995). Zebra striped network file system. CMG Transactions, 85-105.
  More info

  Abstract: Zebra is a network file system that increases throughput by striping the file data across multiple servers. Rather than striping each file separately, Zebra forms all the new data from each client into a single stream, which it then stripes using an approach similar to a log-structured file system. This provides high performance for writes of small files as well as for reads and writes of large files. Zebra also writes parity information in each stripe in the style of RAID disk arrays: this increases storage costs slightly, but allows the system to continue operation while a single storage server is unavailable. A prototype implementation of Zebra, built in the Sprite operating system, provides 4-5 times the throughput of the standard Sprite file system or NFS for large files and a 15-300% improvement for writing small files.
• Drapeau, A. L., Shirriff, K. W., Hartman, J. H., Miller, E. L., Seshan, S., Katz, R. H., Lutz, K., Patterson, D. A., Lee, E. K., Chen, P. M., & Gibson, G. A. (1994). RAID-II: A high-bandwidth network file server. Conference Proceedings - Annual International Symposium on Computer Architecture, ISCA, 234-244.
  More info

  Abstract: In 1989, the RAID (Redundant Arrays of Inexpensive Disks) group at U. C. Berkeley built a prototype disk array called RAID-I. The bandwidth delivered to clients by RAID-I was severely limited by the memory system bandwidth of the disk array's host workstation. We designed our second prototype, RAID-II, to deliver more of the disk array bandwidth to file server clients. A custom-built crossbar memory system called the XBUS board connects the disks directly to the high-speed network, allowing data for large requests to bypass the server workstation. RAID-II runs Log-Structured File System (LFS) software to optimize performance for bandwidth-intensive applications. The RAID-II hardware with a single XBUS controller board delivers 20 megabytes/second for large, random read operations and up to 31 megabytes/second for sequential read operations. A preliminary implementation of LFS on RAID-II delivers 21 megabytes/second on large read requests and 15 megabytes/second on large write operations.
• Hartman, J. H., & Ousterhout, J. K. (1993). Zebra striped network file system. Operating Systems Review (ACM), 27(5), 29-43.
  More info

  Abstract: Zebra is a network file system that increases throughput by string file data across multiple servers. Rather than striping each file separately, Zebra forms all the new data from each client into a single stream, which it then stripes using an approach similar to a log-structured file system. This provides high performance for large files. Zebra also writes parity information in each stripe in the style of RAID disk arrays; this increases storage costs slightly but allows the system to continue operation even while a single storage server is unavailable.
• Baker, M. G., Hartman, J. H., Kupfer, M. D., Shirriff, K. W., & Ousterhout, J. K. (1991). Measurements of a distributed file system. Proceedings of the 13th ACM Symposium on Operating Systems Principles, SOSP 1991, 198-212.
  More info

  Abstract: We analyzed the user-level file access patterns and caching behavior of the Sprite distributed file system. The first part of our analysis repeated a study done in 1985 of the: BSD UNIX file system. We found that file throughput has increased by a factor of 20 to an average of 8 Kbytes per second per active user over 10-minute intervals, and that the use of process migration for load sharing increased burst rates by another factor of six. Also, many more very large (multi-megabyte) files are in use today than in 1985. The second part of our analysis measured the behavior of Sprite's main-memory file caches. Client-level caches average about 7 Mbytes in size (about one-quarter to one-third of main memory) and filter out about 50% of the traffic between clients and servers. 35% of the remaining server traffic is caused by paging, even on workstations with large memories. We found that client cache consistency is needed to prevent stale data errors, but that it is not invoked often enough to degrade overall system performance. © 1991 ACM.

Proceedings Publications

• Choi, I., Nelson, J., Peterson, L., & Hartman, J. H. (2019, September). SDM: A Scientific Dataset Delivery Platform. In eScience.
• Choi, I., Ponsero, A., Youens-Clark, K., Bomhoff, M., Hurwitz, B. L., & Hartman, J. H. (2018, Spring). Libra: Improved Partitioning Strategies for Massive Comparative Metagenomics Analysis. In 9th Workshop on Scientific Cloud Computing.
• Lewis, R., & Hartman, J. H. (2015). Accordion: multi-scale recipes for adaptive detection of duplication. In Proceedings of the 7th USENIX Conference on Hot Topics in Storage and File Systems, 13--13.
• Peterson, L., Baker, S., De Leenheer, M., Bavier, A., Bhatia, S., Nelson, J., Wawrzoniak, M., & Hartman, J. (2015). XOS: An Extensible Cloud Operating System. In ACM International Workshop on Software-Defined Ecosystems (BigSystem15).