November 4, 2016

Bonsai Beta: Welcome to Private Beta and Reinforcement Learning

Bonsai Beta: Welcome to Private Beta and Reinforcement Learning

WARNING: THIS GUIDE IS NOW OUT OF DATE. PLEASE FOLLOW DIRECTIONS IN OUR GETTING STARTED GUIDE INSTEAD.

Now that we're opening the Private Beta to community users, it's time to talk about connecting to the AI engine and teaching it a task. There are many ways to train an AI engine, and depending on the task you're training for, you might choose a simulator, a generator, or a dataset. We're big fans of reinforcement learning here at Bonsai, so let's make a brain and connect a simulator to it.

What's a simulator?

First, let's talk about what a simulator is and does, so that we can understand why it's the best choice for training a goal-directed task that requires a series of actions. In data or generator based learning, each iteration is atomic - there is no relation to the previous iteration. Simulators can be modeled after real-world challenges like FlightGear, video games such as Breakout, or board games like Go.  A simulator will have the following features:

The Web Experience

The NEW BRAIN Button

When first redeeming our Beta Program Invitation, you'll be directed to the Dashboard page, which has a handy button for creating a BRAIN in the upper right corner. This makes it easy to get right into the business of creating and training a BRAIN.

What You'll Need:

We'll start with assuming you've already installed Python, Git, and Pip. Mac users may want to use HomeBrew or download Python directly from python.org so they can perform the following steps on user-land Python rather than system Python. You'll also need your Access Key, which is currently limited to select participants in the private beta. To get one, come visit us in the Forums and let us know why you're excited to try out Bonsai!

How To Do It:

When you redeem your  access key, you're treated to a quick tour of the Bonsai web experience and then introduced to the Dashboard. Our Documentation has a great guide to using the CLI to set up a BRAIN, but if you want to just dive right in from the web page, read on for instructions.

NOTE: Only enter lines in gist links that start with $ into the terminal

Step 1:

The dashboard has a NEW BRAIN button, pictured above. Click on it to start the BRAIN creation process, where you're provided a form to give your brain a name and description. Name can contain alpha, numerals, or hyphens; the description is arbitrary and should contain human-readable text that explains your intent for the BRAIN.

Step 2:

A BRAIN will spawn immediately, and the quick-start guide will appear. This walks you through installing the Bonsai python components to your local machine using pip, so go to your terminal to get it set up. I've made a gist of the pip install step, as well as the results of bonsai configure for reference. Your Access Key can be found in your User Settings, available from your user profile in the upper right corner of the page.

Step 3:

Let's use the caret to collapse the Quick Setup panel and scroll down to the training graph. At this point, there's no data in the chart - we haven't started training yet! We'll need a simulator, so let's git clone the mountain car gym from Github and enter its folder.

 

Next up is the “How to Train mountaincar-brain” section.

Step 4:

We get started by loading our inkling file into the BRAIN from the terminal using bonsai brain load which outputs the following:

 
Step 5:

Almost there! Time to tell the AI engine to prepare a new BRAIN version for training. We do this with the bonsai brain train command.

 
Step 6:

Everything is in place, it's time to start the simulator and test things out. To do this, call Python and then the simulator file, like so:

 

Note that we use headless here to indicate we don't need to see a graphical display from the sim; if you'd like to see it and watch the simulator learn, omit this option.

Step 7:

The simulator will start right away, and the AI engine will calibrate to it for the first thousand cycles (a minute or two). Once it's got a bead on it and it's ready to go, you'll see the graph start building in the center of the page. Training mountaincar can take a few hours, so it's best to find some other way to occupy your time. The load on your local machine should be trivial, so it's fine to go back to development or any other kind of work.

PROTIP: Make sure your computer doesn't fall asleep and drop its link to the BRAIN!

Step 8:

It takes about 1500 episodes to train the BRAIN correctly. Our target is an average reward of -195 or higher over 100 consecutive episodes. Once we've gotten to this level of performance, CTRL-C to stop the simulator and proceed to Prediction.

Congratulations!

You've trained a BRAIN! Follow Bonsai at @BonsaiAI on Twitter for more updates, or come visit our forums to join the community.

Subscribe to our newsletter to stay up-to-date on our latest product news & more!

SUBSCRIBE NOW