Internet of Robotic Things -- University of Washington

Winter, 2021

Tuesdays / Thursdays, 11:30am-12:50pm (Pacific time), Zoom

Instructor: Shyam Gollakota
email: gshyam@cs.washington.edu

TA: Vikram Iyer
email: vsiyer@uw.edu

Description: The future of the Internet will include sensors and Internet-enabled devices with robotic actuators. This has the potential to transform multiple sectors in the economy by enabling various applications including smart farming without farmers, underwater robotics, environmental conservation, global health and space robotics. This course will have invited lectures from researchers across these areas to understand the state of research in these fields and potential open problems for computing, networking and sensing researchers.

Prerequisites: None

Topics

Future farming with the Internet of things
Using drones to address Malaria
Soft low-power robotics
Tracking sensors for underwater robotics
Robotic sampling of phytoplankton and Penguins
Underwater robotics to study polar ice
Space robotics for science and space exploration
Satellite based Internetworking
Powering insect-scale wireless robotics

Grading

Students will be evaluated based on their participants in the lectures and answering questions about each lecture.

Schedule

Date	Speaker	Info
1/5	Shyam Gollakota	Introduction
1/7	Vikram Iyer	Wireless sensing and robotics at insect scale
		Enabling mobility for small wireless devices is challenging, but this is an area where nature excels. This talk presents recent work on bio-inspired and bio-integrative robot and sensor systems. I will begin by discussing challenges and solutions for wireless power transfer that enabled the first liftoff of an insect-scale aerial vehicle. While man-made robots at insect scale are limited by battery energy density, small insects like bees can fly for hours. This includes Living IoT, a first of its kind platform to include sensing, computation, backscatter communication, and localization in a package small enough to fly on the back of a live bumblebee. I will also discuss how we can design more complex sensing systems like wireless cameras that are small enough to be carried by a live beetle, and how these technologies can be adapted to track invasive “murder” hornets.
1/12	Ivan Masmitja	Acoustic tracking by networked moored hydrophones and underwater vehicles in Nephrops no take zones
		Knowing the displacement capacity and mobility patterns of fished marine resources is pivotal to establish effective conservation management strategies in marine ecosystems. Accurate behavioral information of deep-sea fished ecosystems is necessary, but currently scarce, to establish the sizes and adequate locations of marine protected areas within the framework of large international societal programs (e.g. European Community H2020, as part of the Blue Growth economic strategy). A breakthrough in the autonomous capability of mobile platforms to deliver data on animal behavior beyond traditional fixed platform capabilities (e.g. cabled observatories) is overcoming these limitations. Here, we present useful example of that potential in relation to the implementation of autonomous underwater vehicles (AUVs) and remotely operated vehicles (ROVs) as an aid for acoustic long-baseline localization systems for autonomous tracking of Norway lobster (Nephrops norvegicus), one of the key resources exploited in European waters. We reported the outcomes of that monitoring in combination with seafloor moored acoustic receivers to detect and track the movements of 33 tagged individuals at 400 m depth over more than three months. We identified best procedures to localize both the acoustic receivers and the tagged-lobsters, based on cutting-edge algorithms designed for off-the-self acoustic tags identification. These procedures represent an important step forward for prolonged, in situ monitoring of deep-sea benthic animal behavior at meter spatial scales.
1/14	Hanumant Singh	Mapping for Field Robotics: Some thoughts on where we are, where we came from and where we are going
		This talk examines how high resolution mapping in field robotics has evolved over the last 25 years. Looking at examples from expeditions around the world in the pursuit of Marine Archaeology, Marine Geology, Fisheries Science, and Polar Studies, we look at the role of SLAM and other methodologies and where they work and how they are deficient for mapping. We use this to present some of our recent work in light fields for robotics to show how we might be able to address these deficiencies.
1/19	Herbert Shea	Softer Faster Better Stronger: Electrically-driven Elastomer Actuators Our research at EPFL-LMTS centers on mm- to cm-scale elastomer-based actuators driven by electrostatic forces. We aim for high energy density and high efficiency compliant actuators that avoid pumps or compressors. Using examples from our work in soft robotics and wearable haptics, I will illustrate how we addressed several key limitations of directly electrically-driven soft actuators, including obtaining high forces (16 N holding force from a 1 g device), high speeds (5 kHz), complex motion, and reducing drive voltage to 300 V, a level at which we can use SMD components for very compact control electronics. This enabled us to make fast untethered soft robots, robust yet sub-mm thick wearable haptic interfaces, high-force textile clutches for VR gloves, and compliant grippers able to delicately manipulate fruit and vegetables. Our ongoing work is aimed at embedding intelligence into these soft machines.
1/21	Aidan Fitzpatrick	A Photoacoustic Airborne Sonar System (PASS) for Underwater Remote Sensing and Imaging High-resolution imaging and mapping of the ocean and its floor has been limited to around 5% of the global waters due to technological barriers. Whereas sonar is the primary contributor to existing underwater imagery, the water-based system is limited in spatial coverage due to its low imaging throughput. On the other hand, aerial synthetic aperture radar systems have provided high-resolution imaging of the entire earth’s landscapes but are incapable of deep penetration into water. This talk presents a system which bridges the gap between electromagnetic imaging in air and sonar imaging in water through the laser-induced photoacoustic effect and high-sensitivity airborne ultrasonic detection. We have coined this a Photoacoustic Airborne Sonar System (PASS). Mounting PASS on an airborne vehicle such as a drone or helicopter could allow this system to perform high-throughput imaging of underwater in large-scale deployment.
1/26	Kunal Shah	Multi-Agent Coverage Path Planning: Wildlife monitoring and Beyond The ubiquity in aerial robotic platforms for field applications have allowed users in many different disciplines to gather large amounts of data without requiring any specialized piloting training. Autonomous survey and photogrammetry applications involving coverage of a specified area require solving a path planning problem that routes agents to obtain coverage over an area. Many current solutions for aerial coverage are designed for a single agent and do not consider take-off and landing location constraints which are ever-present in many field operations, especially those involving wildlife monitoring. In this work we present a novel multi-agent coverage path planning algorithm based on solving sequential satisfiability modulo theory (SMT) instances. We also present a case study in which we implement our method to survey Adelie penguin colonies in Antarctica.
1/28	Néstor Pérez-Arancibia	The challenging path to creating fully autonomous and controllable microrobots Biological machines, in particular insects, still surpass their robotic counterparts in almost every aspect, including power conversion, actuation, sensing, and control. For instance, the most advanced insect-scale microrobots have yet to achieve the capabilities observed in honeybees and beetles. No subgram robot has been reported to autonomously complete tasks that are challenging or useful for humans, and the vision of creating truly autonomous artificial insects will become a reality only once several of the long-standing grand challenges in robotics are overcome. Namely, new methods for microfabrication, actuation, and control must be developed along with the introduction of new sources of power, mechanisms, structural microcomponents, and actuation materials. In this talk, I will present recent results that represent breakthroughs in power, actuation, and control at the subgram scale, which enabled the creation of three new microrobots with unique and important characteristics from the autonomy perspective. First, I will present the development and control system of RoBeetle, an 88-mg crawling microrobot which is fully autonomous from the energy and control perspectives. To date, RoBeetle is the lightest fully autonomous crawling robot ever created, and one of the smallest. RoBeetle is driven by a novel micro artificial muscle that combines the high work densities of shape-memory alloys (SMAs) with the capability to use sources of power with high energy densities such as hydrocarbon fuels, hydrogen and methanol. Next, I will present the design, fabrication, and control of Bee+, a 95-mg flying robot which is the first, and to this day only, fully controllable four-winged artificial insect driven by four independent actuators, weighing less than 100 mg. Last, I will present SMALLBug, the lightest (30 mg), smallest and fastest microrobot driven by an SMA-based actuator ever created. The key innovation that made the creation of this microrobot possible was the introduction of a new ultralight SMA actuator that weighs 6 mg only. This actuator exhibits the high work density characteristic of SMA materials while achieving high frequencies of operation (up to 30 Hz).
2/2	Trygve Fossum	Searching for Information in the Ocean - Adaptive Sampling of Dynamic Processes in Coastal Areas and Fjords using Autonomous Underwater Vehicles
		The coastal ocean and the upper water column are domains where the need for more sophisticated robotic sampling approaches is critical. Spatial and temporal variability, in addition to episodic events, makes traditional observation practices prone to undersampling. Robotic vehicles have become a critical tool for studying the under-sampled coastal ocean. This has led to new paradigms in ocean sensing, where the combination of agility, reactivity, and persistent presence makes autonomous robots ideal for targeted sampling of elusive, episodic events such as plankton communities. We present methods and results from field experiments using an autonomous underwater vehicle (AUV) with embedded algorithms that focus sampling on features in three dimensions. This was achieved by combining Gaussian process (GP) modeling with onboard robotic autonomy, allowing volumetric measurements to be made at fine scales. Special focus was given to the patchiness of phytoplankton biomass, measured as chlorophyll a (Chla), an important factor for understanding biogeochemical processes, such as primary productivity, in the coastal ocean. The observations from the experiment emphasize the importance of autonomy for sampling dynamic phenomena. The inclusion of embedded autonomy such as the approach demonstrated here forms a basis for enabling larger observation networks and multi-coordination of robotic arrays.
2/4	Deepak Vasisht	FarmBeats: An AI and IoT Platform for Data-driven Agriculture Data-driven techniques help boost agricultural productivity by increasing yields, reducing losses, and cutting down input costs. However, these techniques have seen sparse adoption owing to high costs of manual data collection and limited connectivity solutions. In this talk, I will present FarmBeats, an end-to-end IoT platform for agriculture that enables seamless data collection from various sensors, cameras and drones. FarmBeats’s system design explicitly accounts for weather-related power and Internet outages, and significantly reduces the need for dense sensor deployment on the farm by leveraging techniques from machine learning and computer vision. FarmBeats is currently available for public use as Azure FarmBeats, has been piloted in several countries, and was highlighted by Satya Nadella as one of the ten inventions that inspired him in 2017.
2/9		No class
2/11	Luca Mottola	Beyond the Cloud(s): Building Low-power Software for Nano-satellites Ten years ago, I was working on disconnected operations for wireless sensor networks in applications such as wildlife monitoring. With the advent of all sorts of pervasive long-range communication networks, the potential issue of not being able to connect to the Internet back-end quickly faded away. Now, I find myself dealing with the same kind of disconnected operations, again on a low-power embedded device, which is however orbiting into space. This talk tells the challenges, solutions, and lessons learned while building software for nano satellites. We launched three such satellites and one is still operational. In the process of doing so, I learned how one’s research is to be valued in the near- and in the long-term, and how good system work is to be crafted so it lives past its contemporary technologies.
2/16	Nirupam Roy	Acoustic Perception in IoT: Opportunities and Perils The recent proliferation of acoustic devices is leading to a sensing ecosystem around us. It is opening up new opportunities for human-centric applications as well as efficient perception techniques for cyber-physical systems. However, if not designed carefully, acoustic-IoT can also leave security/privacy loose ends. My research focuses on developing hardware-software building blocks that enable new capabilities for this emerging future and investigate possible security-privacy gaps. In this talk, I will sample some of my recent and past projects. For instance, I will elaborate on our SWaP-friendly acoustic situational awareness system for aerial robots. Next, I will draw attention to various acoustic leakages and threats that arrive with these sensor-rich environments. I will conclude this talk with a glimpse of my ongoing and future projects targeting a stronger convergence of sensing, computing, and communications in tomorrow’s IoT, cyber-physical systems, and healthcare technologies.
2/18	Blake Hannaford	Surgical Robotics via Internet: What could go wrong? The vision of a remotely operated surgical robot is surprisingly old. The first remotely operated surgery on a human was performed by Dr. Jacques Marescaux between New York and Paris in 2001 via a hardened ISDN connection. Internet-based approaches must confront the realities and stochastic guarantees of the modern internet (I'm looking at you Zoom calls). Although bandwidth and latency of the internet are now sufficient for remote surgery, guaranteeing a safe outcome for the patient requires robust automation added to teleoperation which can guarantee the robot and patient go into a contextually dependent safe state upon loss or degradation of the connection. Most of the cybersecurity issues are similar to other high stakes domains like banking, but there are a few that are especially important to consider for surgery.
2/23	Josh Smith	Non-contact manipulation I will describe my group's recent work on non-contact manipulation, in which an array of ultrasound fields are used to levitate small objects. I will describe a novel robotic system that is able to lift an object from a surface (without touching it), transport it to a target location, and release the object. I will also demonstrate controlling the position of the object by adjusting the phases of the ultrasound transducers. Then I will discuss a set of future challenges for non-contact manipulation including manipulating powders, cleaning, non-contact 3D printing, and others.
2/25		No class
3/2	Karthik Sundaresan	Untethering 5G Networks for Ubiquitous Connectivity Advances in mobile (cellular) networks have ushered in an era of abundant connectivity. However, the stationary and expensive nature of their deployment has limited their ability to provide true "ubiquitous" connectivity under the 5G vision - especially to areas where connectivity is sparing or nonexistent (e.g. rural areas), has been compromised (e.g. disasters), or demands are extreme (e.g. venues/hotspots). The recent advances in un-manned aerial vehicle (UAVs) technology have the potential to change the landscape of wide-area wireless connectivity by bringing a new dimension - "mobility" to the cellular network infrastructure itself. By deploying base stations on each of the UAVs, service providers can now deploy and tear-down these cellular networks "in the sky" in an on-demand and flexible manner. This allows them to supplement static mobile networks in areas where additional connectivity is needed, or provide stand-alone connectivity in areas where existing mobile networks are either absent or compromised. However, realizing this vision of deploying heavy-weight cellular networks (e.g. LTE) on light-weight, resource-constrained platforms such as UAVs, faces several formidable challenges both in design and deployment. This is complicated by the complex nature of cellular networks that involve multiple interacting components - radio access network (RAN), evolved packet core (EPC) network and backhaul transport network. In this talk, I will present our system "SkyLiTE"-- one of the first efforts to design and deploy an on-demand, end-to-end, multi-cell LTE network (on UAVs) that can self-configure itself in the sky. I will discuss how SkyLiTE brings together several advances in wireless networking research to re-architect the various components (RAN, core and backhaul) of a cellular network, thereby making it deployable on challenging UAV platforms in highly dynamic environments. The validation of SkyLiTE in real-world deployments shows the potential for contributing to a new era of mobile networks that can fly and make not just connectivity but also sensing ubiquitous.
3/4	Tommaso Melodia	Toward Autonomous, Software-Defined Networks of Wireless Drones Recent advances in drone technologies are making it possible for drones to transport goods, monitor disaster areas, and bring various forms of relief, connectivity, and assistance to areas that are otherwise difficult to access. This talk will cover our recent work on developing autonomous, programmable, and optimized wireless networks of unmanned aerial vehicles in a number of different scenarios. We will discuss applications of drones to augment cellular connectivity while carrying software-defined base stations, or to stream live video in cellular networks. We will then cover applications of self-optimizing networks of drones in disaster and in tactical scenarios, and discuss open research challenges that need to be solved to enable true seamless and programmable connectivity for wireless networks of drones.
3/9	Cédric Corpa de la Fuente	The CubeRover for Lunar Mobility as a Service The CubeRover is a first of its kind ultra-light, modular, and scalable commercial rover. Com-bined with commercial payload services available in CLPS, it offers a low-cost onramp to the Moon for payload developers globally. CubeRovers utilize a methodology akin to CubeSats in that they support diverse instrument packages in a standard form factor. CubeRovers leverage the internationally recognized CubeSat sizing method to define the payload volume and carrying capacity. Larger versions of the rover utilize the same structural, power, thermal, avionics, and software systems to minimize re-engineering costs. CubeRovers are highly customizable to meet the needs of precise payload demands.