{"_id":"563fc7641594380d009c1a61","user":"5503e897e508a017002013bd","__v":7,"version":{"_id":"563fc7631594380d009c1a5c","project":"5503ea178c5e913700362c70","__v":2,"createdAt":"2015-11-08T22:06:27.279Z","releaseDate":"2015-11-08T22:06:27.278Z","categories":["563fc7641594380d009c1a5d","563fc7641594380d009c1a5e","563fc7641594380d009c1a5f","5654ff257b89070d00f96386"],"is_deprecated":false,"is_hidden":false,"is_beta":true,"is_stable":true,"codename":"","version_clean":"0.0.9","version":"0.0.9"},"project":"5503ea178c5e913700362c70","category":{"_id":"563fc7641594380d009c1a5e","project":"5503ea178c5e913700362c70","version":"563fc7631594380d009c1a5c","__v":6,"pages":["563fc7641594380d009c1a61","563fc7641594380d009c1a62","563fc7641594380d009c1a63","563fc7641594380d009c1a64","563fc7641594380d009c1a65","5654d9907b89070d00f96354","5654d9dc055aa40d006f0c29","5654da6cdd82540d00278ae4","5654dab7e0d82b0d00ab572d","5654daf4d667280d0025298d"],"sync":{"url":"","isSync":false},"reference":false,"createdAt":"2015-03-15T09:53:40.258Z","from_sync":false,"order":2,"slug":"tutorials","title":"Tutorials"},"updates":["55ea2f66ba33bc3500b28ce6","56da3af6337fd11300d6a47f"],"next":{"pages":[],"description":""},"createdAt":"2015-03-16T23:13:19.964Z","link_external":false,"link_url":"","githubsync":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":0,"body":"To get up to speed on deep learning, check out a blog post here: [Deep Learning 101](http://markus.com/deep-learning-101).\n\nI also recommend setting up [Theano to use the GPU](http://deeplearning.net/software/theano/tutorial/using_gpu.html) to vastly reduce training time.\n\nOpenDeep relies on the creation of four classes:\n* Dataset: you will use a dataset object to act as an interface to whatever data you train with. You can use standard dataset wrappers provided (such as MNIST or CIFAR10), or create your own from files or even in-memory if you pass arrays from other packages like Numpy.\n\n* Model: the model defines the computation you want to perform. Models take in Theano variables/expressions and output Theano expressions.\n\n* Loss: you must define the appropriate loss (also known as cost) function that defines how well your model is performing. This function will be minimized during training.\n\n* Optimizer: an optimizer takes a model, loss, and dataset, and trains the model's parameters using minibatches of examples from the dataset. This is a logically separate object to give flexibility to training models as well as the ability to continually train models based on new data.\n\n# Hello World: MLP on the MNIST dataset\nA [Multilayer Perceptron](http://deeplearning.net/tutorial/mlp.html) is a simple feedforward neural network that has a classification layer. In this example, we can quickly construct a model container, add some hidden layers and an output layer, and train the network to classify images of handwritten digits (MNIST dataset).\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"from opendeep.models import Prototype, Dense, Softmax\\nfrom opendeep.models.utils import Noise\\nfrom opendeep.optimization.loss import Neg_LL\\nfrom opendeep.optimization import AdaDelta\\nfrom opendeep.data import MNIST\\nfrom theano.tensor import matrix, lvector\\n\\nprint \\\"Getting data...\\\"\\ndata = MNIST(flatten=True)\\n\\nprint \\\"Creating model...\\\"\\nin_shape = (None, 28*28)\\nin_var = matrix('xs')\\nmlp = Prototype()\\nmlp.add(Dense(inputs=(in_shape, in_var), outputs=512, activation='relu'))\\nmlp.add(Noise, noise='dropout', noise_level=0.5)\\nmlp.add(Softmax, outputs=10, out_as_probs=False)\\n\\nprint \\\"Training...\\\"\\ntarget_var = lvector('ys')\\nloss = Neg_LL(inputs=mlp.models[-1].p_y_given_x, targets=target_var, one_hot=False)\\n\\noptimizer = AdaDelta(model=mlp, loss=loss, dataset=data, epochs=10)\\noptimizer.train()\\n\\nprint \\\"Predicting...\\\"\\npredictions = mlp.run(data.test_inputs)\\n\\nprint \\\"Accuracy: \\\", float(sum(predictions==data.test_targets)) / len(data.test_targets)\",\n      \"language\": \"python\"\n    }\n  ]\n}\n[/block]\n#Passing data from Numpy/Scipy/Pandas/Array\nIf you want to use your own data for training/validation/testing, you can pass any array-like object (it gets cast to a numpy array in the code) to a Dataset like so:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"# imports\\nfrom opendeep.data import NumpyDataset\\nimport numpy\\n\\n# create some fake random data to demonstrate creating a Dataset\\n# train set\\nfake_train_inputs = numpy.random.uniform(0, 1, size=(100, 5))\\nfake_train_targets = numpy.random.binomial(n=1, p=0.5, size=100)\\n# valid set\\nfake_valid_inputs = numpy.random.uniform(0, 1, size=(30, 5))\\nfake_valid_targets = numpy.random.binomial(n=1, p=0.5, size=30)\\n# test set (showing you can mix and match the types of inputs - as long as they can be cast to numpy arrays\\nfake_test_inputs = [[0.1, 0.2, 0.3, 0.4, 0.5],\\n               [0.9, 0.8, 0.7, 0.6, 0.5]]\\nfake_test_targets = [0, 1]\\n\\n# create the dataset!\\n# note that everything except for train_inputs is optional. that would be bare-minimum for an unsupervised model.\\ndata = NumpyDataset(train_inputs=fake_train_inputs, train_targets=fake_train_targets,\\n                     valid_inputs=fake_valid_inputs, valid_targets=fake_valid_targets,\\n                     test_inputs=fake_test_inputs, test_targets=fake_test_targets)\\n# now you can use the dataset normally when creating an optimizer like other tutorials show!\",\n      \"language\": \"python\"\n    }\n  ]\n}\n[/block]\n\n#Summary\nCongrats, you just:\n* set up a dataset (MNIST or an array from memory)\n* instantiated a container model and added layer models to form a neural network\n* trained it with an AdaDelta optimizer using negative log-likelihood cost\n* and predicted some outputs given inputs (and computed accuracy)!\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/CPkcSw6IRkdzWekfB5Kl_the_carlton.gif\",\n        \"the_carlton.gif\",\n        \"480\",\n        \"340\",\n        \"#4e3126\",\n        \"\"\n      ]\n    }\n  ]\n}\n[/block]","excerpt":"Here you will learn the basic components for running a deep learning model.","slug":"tutorial-first-steps","type":"basic","title":"First Steps"}

First Steps

Here you will learn the basic components for running a deep learning model.

To get up to speed on deep learning, check out a blog post here: [Deep Learning 101](http://markus.com/deep-learning-101). I also recommend setting up [Theano to use the GPU](http://deeplearning.net/software/theano/tutorial/using_gpu.html) to vastly reduce training time. OpenDeep relies on the creation of four classes: * Dataset: you will use a dataset object to act as an interface to whatever data you train with. You can use standard dataset wrappers provided (such as MNIST or CIFAR10), or create your own from files or even in-memory if you pass arrays from other packages like Numpy. * Model: the model defines the computation you want to perform. Models take in Theano variables/expressions and output Theano expressions. * Loss: you must define the appropriate loss (also known as cost) function that defines how well your model is performing. This function will be minimized during training. * Optimizer: an optimizer takes a model, loss, and dataset, and trains the model's parameters using minibatches of examples from the dataset. This is a logically separate object to give flexibility to training models as well as the ability to continually train models based on new data. # Hello World: MLP on the MNIST dataset A [Multilayer Perceptron](http://deeplearning.net/tutorial/mlp.html) is a simple feedforward neural network that has a classification layer. In this example, we can quickly construct a model container, add some hidden layers and an output layer, and train the network to classify images of handwritten digits (MNIST dataset). [block:code] { "codes": [ { "code": "from opendeep.models import Prototype, Dense, Softmax\nfrom opendeep.models.utils import Noise\nfrom opendeep.optimization.loss import Neg_LL\nfrom opendeep.optimization import AdaDelta\nfrom opendeep.data import MNIST\nfrom theano.tensor import matrix, lvector\n\nprint \"Getting data...\"\ndata = MNIST(flatten=True)\n\nprint \"Creating model...\"\nin_shape = (None, 28*28)\nin_var = matrix('xs')\nmlp = Prototype()\nmlp.add(Dense(inputs=(in_shape, in_var), outputs=512, activation='relu'))\nmlp.add(Noise, noise='dropout', noise_level=0.5)\nmlp.add(Softmax, outputs=10, out_as_probs=False)\n\nprint \"Training...\"\ntarget_var = lvector('ys')\nloss = Neg_LL(inputs=mlp.models[-1].p_y_given_x, targets=target_var, one_hot=False)\n\noptimizer = AdaDelta(model=mlp, loss=loss, dataset=data, epochs=10)\noptimizer.train()\n\nprint \"Predicting...\"\npredictions = mlp.run(data.test_inputs)\n\nprint \"Accuracy: \", float(sum(predictions==data.test_targets)) / len(data.test_targets)", "language": "python" } ] } [/block] #Passing data from Numpy/Scipy/Pandas/Array If you want to use your own data for training/validation/testing, you can pass any array-like object (it gets cast to a numpy array in the code) to a Dataset like so: [block:code] { "codes": [ { "code": "# imports\nfrom opendeep.data import NumpyDataset\nimport numpy\n\n# create some fake random data to demonstrate creating a Dataset\n# train set\nfake_train_inputs = numpy.random.uniform(0, 1, size=(100, 5))\nfake_train_targets = numpy.random.binomial(n=1, p=0.5, size=100)\n# valid set\nfake_valid_inputs = numpy.random.uniform(0, 1, size=(30, 5))\nfake_valid_targets = numpy.random.binomial(n=1, p=0.5, size=30)\n# test set (showing you can mix and match the types of inputs - as long as they can be cast to numpy arrays\nfake_test_inputs = [[0.1, 0.2, 0.3, 0.4, 0.5],\n [0.9, 0.8, 0.7, 0.6, 0.5]]\nfake_test_targets = [0, 1]\n\n# create the dataset!\n# note that everything except for train_inputs is optional. that would be bare-minimum for an unsupervised model.\ndata = NumpyDataset(train_inputs=fake_train_inputs, train_targets=fake_train_targets,\n valid_inputs=fake_valid_inputs, valid_targets=fake_valid_targets,\n test_inputs=fake_test_inputs, test_targets=fake_test_targets)\n# now you can use the dataset normally when creating an optimizer like other tutorials show!", "language": "python" } ] } [/block] #Summary Congrats, you just: * set up a dataset (MNIST or an array from memory) * instantiated a container model and added layer models to form a neural network * trained it with an AdaDelta optimizer using negative log-likelihood cost * and predicted some outputs given inputs (and computed accuracy)! [block:image] { "images": [ { "image": [ "https://files.readme.io/CPkcSw6IRkdzWekfB5Kl_the_carlton.gif", "the_carlton.gif", "480", "340", "#4e3126", "" ] } ] } [/block]