Plotting loss function in Jupyter
Posted in General by Igor Fedorov Tue Apr 26 2016 01:13:03 GMT+0000 (Coordinated Universal Time)·5·Viewed 7,063 times
Is there a way to plot the loss function as a function of iteration number (during training) within the Jupyter notebook?
The master branch lets you plot loss as a function of epoch number during training using Bokeh - see http://www.opendeep.org/docs/monitors-and-live-plotting
If you want to plot per minibatch iteration, see the in-development branch
monitor_customization
https://github.com/vitruvianscience/OpenDeep/blob/monitor_customization/opendeep/monitor/monitor.py#L244Here, you can initialize the Monitor class with the
level
parameter to be 'batch' or 'epoch'.Within the Jupyter notebook, you would have to write your own graphing function given the output of the monitor. If you want the live plot in Bokeh, use the built in
Plot
class here https://github.com/vitruvianscience/OpenDeep/blob/monitor_customization/opendeep/monitor/plot.py as shown in the tutorial above.Let me know if you have more questions!
I would like to go the route of creating my own plotting function in Jupyter, but I'm still confused. Let's use the example (http://www.opendeep.org/docs/monitors-and-live-plotting) as the point of reference. I would like to plot the value of the cost function during training (I'm not particularly concerned about whether it is a function of epoch number or minibatch number). I'm assuming that I would need to first create a MonitorsChannel:
output_channel = MonitorsChannel(name="training_error")
and then pass this to the optimizer:
optimizer.train(monitor_channels=output_channel)
Is this correct? If so, what is the next step?
On a sideonote, can you provide a reference for how to use markdown within the forum?
Yep that is correct! you don't need to make a full MonitorsChannel if you aren't grouping more than one monitor together, you can use the Monitor class directly with optimizer.train(monitor_channels=output_monitor). You can do something like:
from opendeep.monitor import PrintService
output_monitor = Monitor(name="training_error", expression=your_error_calculation_in_theano, out_service=PrintService(name="training_error"), train=True, valid=False, test=False)
optimizer.train(monitor_channels=output_monitor)
This will print out the monitor values to the console (stdOut, wherever Python's print function goes). If you want the values to be stored in a file to graph later, you can use a FileService instead of a PrintService. Details can be seen here: https://github.com/vitruvianscience/OpenDeep/blob/master/opendeep/monitor/out_service.py#L39
And here is a good Markdown cheatsheet: https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet
Best bet might be to use the FileService and have a separate jupyter process loop reading from the file and graphing while the training is happening so the graph gets updated in realtime.
Or to use the builtin Plot to make a Bokeh plot in your browser instead of having to use Jupyter :)
https://github.com/vitruvianscience/OpenDeep/blob/master/opendeep/monitor/plot.py
plot = Plot("training_error", output_monitor, open_browser=True)
optimizer.train(plot=plot)