Davide BrugaliUniversity of Bergamo
on Managing software variability in robotic control systems.

Abstract: Sensing, planning, controlling, and reasoning, are human like capabilities that can be artificially replicated in an autonomous robot as software systems, which implement data structures and algorithms devised on a large spectrum of theories, from mechanics, and control theory to ethology and cognitive sciences. Software plays a key role in the development of robotic systems as it is the medium to embody intelligence in the machine.

In this scenario, the cost of creating new robotics products is significantly related to the complexity of developing software control systems that are flexible enough to easily accommodate frequently changing requirements and to exploit fast evolving technologies.

In various application domains software product line (SPL) development has proven to be the most effective approach to face this kind of challenges. A SPL is a set of software intensive systems that share a common set of features for satisfying a particular market segment’s needs. Each new application is built from the SPL repository of common software assets (e.g. architectural models, software components)

This course will illustrate the basic concepts of SPL and exemplify them in the context of robotics. More specifically the six lectures will focus on:

  • Lecture 1: Analysis of the software development problem in robotics and illustration of the benefits of the SPL approach
  • Lecture 2: Techniques for Domain Analysis and Modelling
  • Lecture 3: Case Study of Domain Analysis: Robot Navigation
  • Lecture 4: Review of Software Variability Realization Techniques
  • Lecture 5: Case Study of variability implementation: ROS MoveIt!
  • Lecture 6: Dealing with Non-Functional Properties in SPL

Rodolphe Gelin, Softbank Robotics
on Humanoid robots for a social man-machine interaction.

Abstract: With NAO and Pepper, SoftBank Robotics proposes the two first affordable humanoid robots dedicated to the social interaction with people. This represents a major technological challenge considering the current state of the art in humanoid robotics. But, mainly because of science fiction movies, the expectations of the customers are still much higher than what science and technology are able to provide even in the best research laboratories. How is it possible to avoid deception while gaining the acceptability of the public ? One of the solution is to dedicate the a priori universal humanoid robot to very specific tasks. By focusing on restricted applications, it will be possible to provide efficient humanoid robots, accepted for the service they are supposed to bring and not being feared for bad reasons. As science and technology progress, the service brought by humanoid robots will become closer and closer to the expectations but it will give time to the society to adapt the role of the robots at our side. 

The 6 lectures will focus on 

  • The robotic vision of SoftBank Robotics
  • The existing humanoid robots
  • Current applications of humanoid robots
  • Technological challenges in physical interaction
  • Technological challenges in social interactions
  • The non technological challenges

Ashish Kapoor, Microsoft Research
on Safe Machine Learning for Robotic Systems.

AbstractMachine Learning is one of the key component that enables systems that operate under uncertainty. For  example,  robotic systems  might employ  sensors  together with  a  machine  learned system  to  identify  obstacles.  However, such data driven system are far from perfect and can result in failure cases that can jeopardize safety. In this series of lectures we will explore ideas that  aim to preserve safety invariants despite the uncertainties in the environment arising due to incomplete information.  We will first describe popular methods for perceiving the world through noisy sensors and machine learning systems, and how they could help us plan under uncertainty. Next, we will in depth study machine learning and intelligence techniques that allow us to encode uncertainty in perception and how they could be combined with Formal Methods to derive plans that have probabilistic safety guarantees. We will consider extensions of these ideas to a sequential decision making framework that considers the trade-off in risk and reward in a near-optimal manner.

Nenad MedvidovicUniversity of Southern California
on Software Architectures of Robotics Systems.

Abstract: Software architecture is a set of principal design decisions about a software system. These decisions are embodied in the system’s components (operational entities that perform computation and/or store data), connectors (entities that facilitate interaction and coordination among components), and configurations (assemblies of components and connectors into system-specific topologies). The architecture of any complex, distributed, and decentralized software system is the key determinant of that system’s critical properties. An important emerging class of such software-intensive systems are robotics systems. Robotics systems exhibit characteristics that argue for a software architectural focus, including a high degree of heterogeneity among constituent subsystems, strict operational requirements dictated by real-time interactions with the physical world, and system complexity that extends beyond a single engineer’s ability to grasp and manage it. This course will focus on five key facets of the link between software architectures and robotics systems:

  1. exploration of the architectural design space and of the effective software design solutions within that space, needed both for the initial system design and subsequent dynamic runtime adaptations;
  2. modeling the distributed software-intensive system that is deployed on a set of robot, and possibly traditional, platforms (as opposed to, e.g., modeling specific robotic algorithms);
  3. analysis of the system architectural models for key properties before constructing and deploying the system and during dynamic adaptations;
  4. traceability of the architectural design-time artifacts, such as components and connectors, to implementation constructs; and
  5. support for heterogeneous robot development and deployment platforms.

Bertrand MeyerPolitecnico di Milano
on Concurrent Object-Oriented Robotics Software: Concepts, Framework and Applications.

Abstract: These lectures will cover robotics software from a software engineering perspective, including:

  • Challenges of building robotics applications
  • Architecture of robotics systems
  • Design patterns for robotics
  • Making robotics applications concurrent
  • Experience with the Roboscoop framework and its applications

Issa NesnasNASA Jet Propulsion Laboratory
on Experiences from robotic software development for research and planetary flight robots.

Abstract: Robotic explorers are complex machines that venture into space flying by planetary bodies and sometimes orbit, enter, land, and traverse their surfaces. These explorers often operate in the harsh environment of space under thermal extremes and with severe constraints. Chief among these constraints are limited computation and memory, restricted communication windows with large latency as well as limited power and energy. Perhaps the most recognized of these surface explorers are the Mars landers and rovers: Viking I and II, Sojourner, Spirit, Opportunity, Phoenix and Curiosity that have provided a wealth of information across decades of exploration, leading to major scientific discoveries about the red planet. Future explorers are anticipated to reach worlds where even less is known about these new environments. These systems have to be architected with reliability and flexibility that enable their remote operation, at times, over long durations. Both robotics and autonomous systems are anticipated to play a critical role in providing the needed robustness and intelligence in operating in these unknown environments.

In this series of lectures, we will focus on robotics software that largely embodies the functional, cognitive and autonomous capabilities of these machines. We will introduce the range and heterogeneity of such robotic explorers and share some of the challenges in architecting and developing their software. We will examine robotic software that was developed for flight and robotic prototypes with the goal of improving reusability, flexibility and maintainability and share some lessons learned. Some of the topics that will be addressed through these lectures include:

  • A discussion of architectural principles for developing autonomous robotic systems
  • Addressing and handling system state and uncertainty
  • Software interoperability across heterogeneous platforms
  • Challenges and techniques for developing reusable robotic software: component design, object-oriented design, design patterns, data flow patterns (synchronous and asynchronous), event-based programming, and finite state machines
  • Handling interoperable models that support both functional and planning components
  • An in-depth look at a rover navigation scenario through the lens of robotic software architecture covering activity planning, scheduling and execution, as well as functional behaviors: perception, mobility, motion planning, and navigation; in addition to models, data structures and devices

Hiroshi “Gitchang” Okuno, Waseda University and Kyoto University
on Open-Sourced Robot Audition Software HARK: Capabilities and Applications.

AbstractRobot audition, i.e., hearing capability with robot’s ears, is critical for successful deployment of a robot to various environments and situations. Main functions of robot audition include sound source localization (SSL), i.e., identifying the direction of arrival of each sound source originate, sound source separation (SSS), i.e., separating each sound source, and recognition of separated sounds, because people and robots usually hear a mixture of sounds. In the 20th century, most robots did not have hearing capabilities even if they had microphones or ears. When a robot interacts with people, it hears targeted voice, environmental sounds and its own motor and movement noise. Most robot systems for social interaction used either a head-set microphone worn by a talker or a microphone placed close to a talker, while only a few studies used microphones of a robot, or ears. In the 21st century, some robots can localize a single sound source and a few can localize multiple sound sources through their own microphones.

We have developed a robot audition open source software “HARK” as a general framework for robot audition applications like “OpenCV” for visual applications. HARK has been ported to several humanoid robots such as SIG2, Robovie, HRP-2, and Hearbo to demonstrate various types of human-robot interactions; listening to three simultaneous meal orders, robot quiz master, and co-player musical robots. HARK currently has more than 90 K downloads.

My lecture will consist of as follows:

  1. Design and implementation of Robot Audition Software “HARK”.
  2. Fundamental functions such as SSL, SSR and interface to automatic speech recognition systems
  3. Analysis of HARK performance and embedded HARK
  4. Software oriented Thermine (electric musical instrument) playing robot
  5. Applications of HARK including human-robot interactions, and
  6. Applications of HARK in wider areas including animal calls and search-and-rescue activities. HARKBird captures sounds with an inexpensive microphone array to localize and separate birdsongs in their natural habitat. Frog chorus is analyzed by using sound-indication devices “Firefly” and demonstrated that Japanese Tree Frogs call alternately in their natural habitat. As search-and-rescue activities, a hose-shaped robot with microphones estimates its 3D-posture by emitting a sound and enhances human voices.

The bibliography of robot audition technologies is available at the URL of


Special guest lecture

Andrey Terekhov, Saint Petersburg State University and LANIT-Terkom
on ERIK Studio – IDE for robotic programming.

Abstract: This talk presents TRIK Studio – an educational programming environment for robotic kits. TRIK Studio uses a visual domain-specific language to program Lego Mindstorms NXT, Lego Mindstorms EV3 and TRIK robotic kits, and it also allows to run a program on a 2D simulator without the need of an actual robot. TRIK Studio is used in various educational organizations across Russia and Europe and in an online course on robotics. It also supports textual programming in various languages including JavaScript, C, Pascal, F#, and supports generation of human-readable code by diagrams. It also has some important usability-related features that enable even children that do not know how to read to program robots. In this talk we discuss main features of TRIK Studio, its differences from similar programming environments, and will perform a demonstration of a tool by programming an actual TRIK robot.