About RL-Glue
What Problem does RL-Glue Solve (Why use RL-Glue)?
Typically, individuals in the reinforcement learning community use their own interface and software framework for evaluating learning methods. These incompatible frameworks make it difficult to share code, collaborate, and compare results with others.
RL-Glue provides a standard interface that allows you to connect agents, environments, and experiment programs together, even if they are written in different languages. This has a number of benefits, such as:
- Program agents and environments in the language of your choice
- Re-use agents and environments that have been created by other members of the reinforcement learning community
- Easily share your own agents and environments, making it easier for people to compare and build on your work
- Teach a class that involves reinforcement learning, leveraging all of the "boring" plumbing provided by RL-Glue
- Extend your crazy simulator/game/problem to work with RL-Glue, making it accessible to the rest of the reinforcement learning community
- Run reinforcement learning agents on your robot, in the language of your choice
Click here to see some of the examples of RL-Glue being used by the community.
The Projects
RL-Glue is both a set of ideas and standards, as well as a software implementation provided through this site. In theory, RL-Glue is a set of common guidelines for the reinforcement learning community to follow to allow us to share and compare agents and environments with greater ease.
The software implementation of RL-Glue is the reusable glue to connect the basic parts of an experiment. RL-Glue is functionally a harness to "plug in" agents and environments and experiment without having to continually rewrite the connecting code.
This project has been growing for a number of years. The most recent release is RL-Glue 3.x (October 2008) is a big step forward for this project. We have written documentation for RL-Glue and related projects, we've used standard tools to build and install RL-Glue, and we hope to publish at the NIPS Workshop on Machine Learning Open Source Software 2008 and to the special Machine Learning Open Source Software track of the Journal for Machine Learning Research.
The RL-Glue project is actually spread over a number of different projects.
RL-Glue Core Project
The RL-Glue Core Project includes software for creating direct-compile project, the stand-alone server for running socket projects, and an overview document that explains the core ideas and vision of the RL-Glue project.
The web page for the RL-Glue Core project is, including links to downloads, project overview, and technical manual are here:
http://glue.rl-community.org/Home/rl-glue
The open-source RL-Glue Core project (http://rl-glue.googlecode.com) is meant to be a robust, stable project that you install once and rarely (if ever) worry about updating.
RL-Glue Extensions
The RL-Glue Extensions project is separate from the core because it is a more diffuse effort that is meant to be agile and updated on a regular basis by various contributors. While extensions to RL-Glue could be written and distributed by anyone, the RL-Glue Extensions project (http://rl-glue-ext.googlecode.com) offers a central place where they are welcome to go.
The extensions that are currently in this project are all Codecs (below), but in the future we imagine the community may wish to contribute other supporting projects into the RL-Glue extension project.
Codecs (Multi-Language Support)
RL-Glue supports interaction between agents, environments, and experiments in two different modes. In direct-compile mode, all three modules are written in C/C++, and they are compiled together into a single executable program.
In the more flexible, socket mode, agents, environments, and experiments using inter-process communication through sockets, either locally on one computer or over the network or Internet. In socket mode, agents, environments, and experiments written in a variety of languages can interact with each other transparently. The language-specific software that allows creations from a particular language to connect to RL-Glue is called a codec. We currently have codecs for: