{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "4jQLbOS7JhxD" }, "source": [ "# PyTorch Introduction(Con't)\n", "\n", "Today, we will be intoducing PyTorch, \"an open source deep learning platform that provides a seamless path from research prototyping to production deployment\".\n", "\n", "This notebook is by no means comprehensive. If you have any questions the documentation and Google are your friends.\n", "\n", "Goal takeaways:\n", "- Automatic differentiation is a powerful tool\n", "- PyTorch implements common functions used in deep learning" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "Rw-LPR3HJhxJ" }, "outputs": [], "source": [ "import torch\n", "import torch.nn as nn\n", "import torch.nn.functional as F\n", "\n", "from mpl_toolkits.mplot3d import Axes3D\n", "import matplotlib.pyplot as plt\n", "\n", "import numpy as np\n", "\n", "torch.manual_seed(446)\n", "np.random.seed(446)" ] }, { "cell_type": "markdown", "metadata": { "id": "vhmk4YPWJhxT" }, "source": [ "# Neural Network Basics in PyTorch\n", "\n", "Let's consider the dataset from hw3. We will try and fit a simple neural network to the data." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "cD2kkX3PJhxT", "outputId": "a184128e-adc3-45f4-cbf4-e76312d0af0c" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEYCAYAAABRB/GsAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAmDUlEQVR4nO3df3RcZ3kn8O/XyiSRIa2SjVjwxIpDCgaMYqvoYOX4nC0JAaehTmaTQjA2XbYsPrQLS1JWrJ3o1A412KCF5OzSLTWFLa2NMQEzdXC6TqhDs5uDvJWRY8VJDAkFORO6mCZKAStElp/9Y2bk0eje+eV773vv3O/nHJ+jmbkzeu9YM8993+d9n5dmBhERSa8FrhsgIiJuKRCIiKScAoGISMopEIiIpJwCgYhIyikQiIiknAKBiEjKKRCIiKScAoGkDskfkbwuot+1lOQRkj8n+Z98jukm+SDJ50l+ieQ2krc1+Pr/l+SyQBstqXOe6waIxBnJHwH4D2b27RZf4mMAHjKzFTWO2QTgB2b2NpLdAI4A+I0GX/+/Avg4gFtabJ+IegQiIbscwLE6x1wH4N7Sz+8DcL+ZTTX4+vsAXEPyla01T0SBQNpUafhnE8nHS0Mu/5PkhR7HvZ7kd0hOkjxG8saKx/4aQA+A+0j+guTHmnz+QQDXAPhc6fmvrXru+SRfANBb+h3jAH4bwN9XHfdpkvmK28Mk/47k+Wb2IoDDAFa39EaJQIFA2ts6FL8grwTwWgBDlQ+SzAC4D8ADAF4B4MMAdpFcCgBm9l4AEwDWmNnLzezTTT7/WgD/G8CHSs//fuXzzewlAFcD+Gnp8V4Ug8LxqvP4FIpX/X0kPwjgegA3l54PAE8AWN7KGyQCKBBIe/ucmZ0ws+cAfALA2qrHBwC8HMB2M3vJzA4C+JbHcX7O9fkAsALAoxW3uwD8vPIAM/tnAHcD+DKK+YQbzOyFikN+XnqeSEsUCKSdnaj4+ccAFlU9vgjACTM7U3VctsHXP9fnA/MDwfMALvI4bgzF3sImMztR9dhFACab+J0icygQSDtbXPFzD4Bnqx5/FsBikguqjitU3K61YUcjz69nOeYGgqMoDmPNItkL4M9Q7BH8vsdrvL7qNUSaokAg7ew/kryM5CUA7gSwp+rxQwBOAfgYyQzJtwBYA+CrFcf8PwCv9nn9Rp5fT3UguB/Ab5VvkMyimIf4IIA/BNBb+j3lxy8E8CYADzbxO0XmUCCQdvYVFBO5PwTwNICtlQ+Wkq1rUJyp8zMA/wPA75nZkxWHbQMwVJoV9J9beL6v0pTPiwFUHv9XAG4g2Uny11AMDJ81s31mdgrAMIr5jrI1AL5jZtW9HZGGUVtVSjsKYCGYMyQ/ieJMonsaOPYQgPeb2WOhN0zallYWi8SMmd3RxLErw2yLpIOGhkREUk5DQyIiKacegYhIyiUyR3DppZfakiVLXDdDRCRRDh8+/DMz666+P5GBYMmSJRgdHXXdDBGRRCH5Y6/7NTQkIpJyCgQiIikXm0BAsoPkGMlvuW6LiEiaxCYQAPgIinXVRUQkQrEIBCQvA/AOAH/hui0iImkTl1lD96C4ybdXHXYAAMkNADYAQE9PTzStEomBdV/4Lh55+rnZ26uuvAS7PnC1wxZJu3HeIyD5OygW2Dpc6zgz22Fm/WbW3909bxqsSNsZyo9jycb9c4IAADzy9HNYsnE/Vtz1APJjzWx9IOLNeSAAsArAjaVqkV8FcC3JnW6bJOLWUH4cO0cmah4zOTWNwXsfVTCQc+Y8EJjZJjO7zMyWAHg3gINmtt5xs0Sc2n2oejdKb9NnDMMHqve6F2mO80AgIvPNNFEM8tnJqRBbImkQl2QxAMDMvgPgO46bIeJUs0M9i7o6Q2qJpEWsAoFI2g3lx7GrTm6gUmYBMbh6aYgtkjTQ0JBITOTHCtg1MgG/QaFM1ae1qzOD4XcuR64vG3rbpL2pRyASE8MHjvsGAQL4wSffEWVzJEXUIxCJiVpJX+UBJEzqEYjExKKuThQ8ggEB3zxAfqyA4QPH8ezkFBZ1dWJw9VINFUnT1CMQiYnB1UvRmemYcx8BrBvo8fxyz48VsGnvOAqTUzAAhckpbNo7rgVm0jQFApGYyPVlse3mXmS7OkEA2a5O3H3rCmzN9XoeP3zgOKamZ+bcNzU9owVm0jQNDYnESK4v2/DQjtcwUq37RfwoEIjEQCtj/R2k5wrkDjKsZkqbUiAQcaw81l8e5imP9QOoGQz8ylA0U55CBFCOQMS5Vsf6sz5TSv3uF/GjQCDimN/6gXrF5LxmGXVmOlRyQpqmQCDimN9isXqLyLxmGW27uVfrCKRpyhGIODa4eumcHAHQ+JV9M7OMRPwoEIg4Vv4i1wphcUWBQCQGdGUvLikQiDikWkESB84DAckLATwM4AIU2/N1M9vstlUi4Wt1/YBI0OIwa+hXAK41s+UAVgC4nuSA2yaJhE+1giQunPcIzMwA/KJ0M1P6p6WR0vZaXT8gErQ49AhAsoPkEQA/BfCgmR3yOGYDyVGSoydPnoy8jSJBa3X9gEjQYhEIzGzGzFYAuAzAm0m+0eOYHWbWb2b93d3dkbdRJGhaGSxxEYtAUGZmkwAeAnC946aIhE4rgyUunOcISHYDmDazSZKdAN4G4FOOmyUSCa0fkDhwHggAvArAl0l2oNhD+ZqZfctxm0REUsN5IDCzowD6XLdDRCStYpUjEBGR6CkQiIiknAKBiEjKOc8RiKTNUH4cuw+dwIwZOkisXbkYW3O9rpslKaZAIBKhofw4do5MzN6eMZu9rWAgrmhoSCRCuw+daOp+kSgoEIhEaMa86yn63S8SBQUCkQh1kE3dLxIFBQKRCK1dubip+1uVHytg1faDuGLjfqzafhD5sUKgry/tRclikQiVE8JhzhrKjxUweO+jmD5THG4qTE5h8N5HAWjnM/FGS+DYZH9/v42OjrpuhkgsrbjrAUxOTc+7v6szgyOb3+6gRRIXJA+bWX/1/eoRiLQZryBQ6/6ky48VMHzgOAqTU+ggMWOGbFcnBlcvVQ+oQQoEIpJYQ/lx7BqZmN3btjz7qjA5hdv3HMHoj5/T+owGKFks0mYuXphp6v6kyo8V5gSBagZg18iEEuUNUCAQaTOb1yxDpmPudNRMB7F5zTJHLQrHpr1HfYNAmQHYsu9YFM1JNAUCkTaT68ti+HeXz+kBvOz89hoFHsqPY2r6TEPHTk5Nq1dQh/NAQHIxyYdIPk7yGMmPuG6TSDv4xYunZ3+enJrG4L2Pts0X4lcOTdQ/qMKd3xwPqSXtIQ6XCacBfNTMvkfyIgCHST5oZo+7bphIUMozW56dnMKiCGa0bNl3bHYdQdn0GcOWfccSP5MmP1bAmSZnvf/ypZlwGtMmnAcCM/sJgJ+Ufv45yScAZAEoEEhbyI8VsGnvOKami19GhckpbNpbvEIN60u5naeQDh847roJbcf50FAlkktQ3L/4kMdjG0iOkhw9efJk5G0TadXwgeOzQaBsanpGX2gtenZyqunnqJJTbbEJBCRfDuAbAG4zs3+pftzMdphZv5n1d3d3R99AkRb5fXG18oXWqFpTRZOeJ1jU1el5f2dmAdYP9Hg+ts7nfimKRSAgmUExCOwys72u2yMSJL8vLr/7g1BrqmjSeyKDq5eiM9Mx577OTAe23XwVtuZ6sX6gZ7aaaweJ9QM9WlRWh/McAUkC+CKAJ8zss67bIxK0a17XPW/hU2emA4Orl4b2O3N9Wdy254jnY2H2RKJQzqv4Jd+35nr1xd8k54EAwCoA7wUwTvJI6b47zOx+d02Kt/xYAXfddwzPnyom/hZmFsCA2XnVFy/MYPOaZYmfHdIO8mMFfONwYU4QIIBb3pQN/f8n29WJgseXfpg9kajk+sJ//9LEeSAws/8D5XJ8VRbU8nOqamHN86emcdueI3OuCDszC7Dt5qv04YmYV6LYADz0ZPgTHgZXL50zWwkIvyciyeQ8EIi3/FgBW/YdC2y639T0Gdy25wg27T2qgBAhvwBeK7AHpfx/XPl3dGEmFmlBiRkFgpgJOgBUKwcEVWWMRrksstf9UfnV6bM9xudPTYe+hkGSR5cHMVJeeBTFop+dIxMYymvZfdhcb1avNQzSCPUIYsTrQxumXSMT6L/8El0ZhsgvYZuNKGHrYg1DGKIu0ZE26hE4NpQfx5KN+7Fk4/6Wx40XZhags4WxXwNw254jWHHXA4lfZBRXfnPeo0rYuljDELRyT7kwOQXD2RId+psNjnoEDg3lx7FzpLkqigBAAutW+i+SKX5wjjZVplebm4ej3pz3sLXDzKFaw1v6ew2GAoFDuw+daOr4rs4MttxYf31A5Rzr/FgBd+w9Om+KabXpM6YPVkhcznl3HYiC4NdTTtrwVpwpEDhQHu+slzAMYiPu8pdQ9d6uXvTBak9JXnxVa/gnScNbcadAEKFmp4Y+ve2GwH731lwv+i+/BB/92qO+AUgfLImbWttMJml4K+6ULI5Is1NDV115SeBtyPVl8Zl3LZ+3ny1Q/EM49dJpXLFxP1ZtP6hEnMRCrc9LUns5caQeQQTyY4WaV+LVVl15CXZ94OpQ2lL+8FTWKurMLMDpMzZ7O4qNU0QkPhQIQlbuCdQLAtmuTjyy8dpI2lQ9Zrxq+8F5CTnNypA4WJhZ4DnRYaFKZQRKgSBEjfYEXE/na5dFR9J+Lsh0eAaCC6rWZjRKC9O8KayGZCg/jtv3HKkbBC5emMG2m3ud/jH6JYkXkMoViFOTp3z2Xva5vxYtTPOnHkHAGpmmCRSnhn7mXctjcTXitegIKNbDUa5AXFoU4J4KWpjmTz2CAK37wnexs4Eg0JnpiE0QAIpf8ttu7vWsiDk1PVNzCp9ImIIs0aEhUH8KBAHJjxXwyNPP1T2ug3Q+FOQl15fFGZ9hrMmpaVUqFSfKFynZrk4QxUkVrX5+2qHuUlhiEQhIfonkT0k+5rotrbrzm/W/KAnEqidQrdYHYtfIhMZSJVL5sQJWbT+I20s77d196wo8svHalj8/rgsAxlksAgGAvwRwvetGtCI/VkDfxx/AL1+qXz563UBPbIMAUHulpgGqYS+RCSOxG2Tvot3EIllsZg+TXOK6Hc1a94XvNjQcBADrB/yrhcZFri87Z6FZNY2lSlTCSuwmue5SmOLSI6iL5AaSoyRHT54Mf+PvepoJAvfcuiL2QaBs85pl8NtEUdNJJSpK7EYrMYHAzHaYWb+Z9Xd3dzttS6OJYaAYBJJ0BZLry2LdQI9nMChPJ1UwkLApsRutxASCOLnrvvrTKYnicFCSgkDZ1lwv7r51he90UuUKJGxK7EZLgaAJ5cSw3xh6WVdnBncnaDjIS63ppOqeSxQuOO/s11PQK/DLM5JUbbcoFslikrsBvAXApSSfAbDZzL7otlVz5ccKGPz6o5ieqb1cLMzKoVELclWnSKPKM4Yqk8UvNrjtaiuvr2q7MekRmNlaM3uVmWXM7LK4BQGgOByUpiAAqHsubtSaMZSE10+iWPQI4qxcrbCR4aB2CgJAe+x3K8kT9owhzUiaT4GgBq8uqpfOTAe23LgsolZFq3LedTko3r7niIKChCbsIUkNec4Xi6GhuPLqQlYjkIrViSrh2z7inigNe0hSQ57zKRD4GMqPe141VMosIO5O2DqBVvmNq6oyabIkIaCHXQpCpSbmozW4j26c9Pf32+joaCivXfygHMVUnVkK2ZQNjVyxcb9vee2kLZpLM69tSYFot0oVd0geNrP+6vuVI6hwNifgHwQ6Mx2pvHrwG1cFgC37jqXu/UgqJUrn0taVRQoEJY3uL5zGIAAUx1VvK5UDrjY51fy2ge1qKD+O3YdOYMYMHSTWrlwcq4WFSpSepfUEZylHgLN/EPWCQAeZuj+QsrSedzOG8uPYOTIx+3c0Y4adIxOx2tRHidKztJ7gLAUCFIc26s0OAoC1KxdH0Jr4unhhpqn702b3oRNN3e+CEqVnaZjsrNQPDQ3lx+sObSwg8J6V8d9PIGyb1yzzLLNhVuxVpfHLpJJfj7JeTzNqqslfpGGys1IbCMrd+Fo6yFhvLRm18vtQvXnN5NR0asdWK3WQnl/6XlVcxb3B1UvnLRhN6zBZKoeG1n3hu3WDABDv/YVdyfVlsfD8+dcPaR1breQ3dJj2IcW40jDZWanrETS6qczFCzOp/INohN8Yar0FeO2uPHQY51lDMlcch8nKU1oLk1Ozvcyw1y2lLhDcsfdo3WOI4ni4ePMbWyWUK9ia69UX/zlI+7z+6imt5aHGsKe2pioQ5McKONVAXfN1Cd1ZLCqDq5fi9j1H5q00NmhxmbQujfP6qwPfqZdO+85gLA+/hvFepCZHUF4wVs/6Ac0OqifXl/UtNzE5NR2rujWSHGmb11/e7Kqy7lO9cveFyalQigWmokfQ6IIx1cxpXLZGyYmwrlqkvbmc1+9iSKqRza68VBYLBILpLcWiR0DyepLHST5FcmPQr99IOWkFgebUmmKXxgU5cu785u+HPa/fRUXW/Fih7tV/PUH2luoGApIPklweyG/zfv0OAH8K4LcBvAHAWpJvCPJ31PpiIorDQQoCzcn1ZX1XFKdxQY6cO1flL6IekioHnlqypc9QvTUoQV10NTI09F8A3EPyRwDuMLOfBPKbz3ozgKfM7IcAQPKrAG4C8HhQv8BvlosWjJ2bzWuWaUGOBMbV1qhRD0nVG6Ho6szMKwnuVz48qIuuuoHAzL4H4BqStwD4XyT3Avi0mQX1LmUBVBZjeQbAyuqDSG4AsAEAenp6mvoFfisI07p4JCiVH9zynOfKKym9t9IsF/P6oy41USvAZBbQc9vbsFdBN5QjIEkAxwH8GYAPA/gByfcG0oIGmdkOM+s3s/7u7u6mnqsVhOHJ9WVnu/TVc541e0iSIOohKb8A00Fi+J3eIxRhf4fV3aGM5CMArgBwDMAIgEMAngTwEQAXmNmGc2oAeTWALWa2unR7EwCY2Ta/54S5Q5k0T7teJVPaF29VivK9qF4vAUQ3QnEuO5RtAPC4zY8YHyb5RABt+wcAryF5BYACgHcDeE8ArysRUTnf5Enj4q1aohyScpULqaWRHEGt3cnfca4NMLPTJD8E4ACADgBfqvM7JWZUzjd5as2USWMgiFrcahyd0zqC8kyfc2Vm95vZa83sSjP7RBCvKdHRrlfJo16cVIrFgjJJNiXjk8fV4i2JJwUCCUSuL4tHNl6Lu29dAQC4fc+RwOuhSHDUi4tWfqyAVdsPhlInKAipqDUk0VACMjnimLBsV0n4XKhHIIFJW/XIpCuvAVnU1YlnJ6cwfOB47K5U20ESPhfqEUhglIBMliRcqSbdUH7ct0pvnD4X6hFIYJSATJYkXKkm2VB+vObe6HH6XCgQSGCUgEwW9eDCteuQfxCI2+dCgUACo2mkyaIeXLhqVe+J2+dCOQIJVNxWTIq/sCtair+4fUYUCERSSlNIw1Nr9tXCTPwGYhQIRFKsOhhoL4m5Wq1Keuc3/Xcg++TNVwXZxEAoEIikmKaQ+mv1vRnKj+OXL/nvQBbH9zV+fRRpK3FfWp92mkLqr9X3ZvehE76PZWOaiFePQEKjq83485sq6rcIKk1anV47U2O6UFwT8eoRSGh0tRl/flNFidoJz6DEucfo9950LczUfF4H6Xk/Ed8LIAUCCY0WLMXf4Oql8PraMiD0gF3uMRYmp2CI317Xg6uXItMx/915/tQ0hvLzk8HloObXI1g30BN4G4PiNBCQfCfJYyTPkJy3j6YkmxYsxV+uLwu/gYywA3bce4y5vixedr736PnOkYk5ASs/VsDg1x/1HFLrILF+oAdbc72htfVcue4RPAbgZgAPO26HhEAlJ5LBL4EZdsBOQo/xhalp38fu2Ht0zs/TM/ND6sULM3h62w2xDgKA40BgZk+YWTzCvwROJSeSwStgE8A1r+sO9fcmocdYqy2nps8gP1ZAfqyAU9NnPI95/pR/IImTxMwaIrkBwAYA6OmJ71ibzKWSE/GX68ti9MfPYdfIxOwwkQH4xuEC+i+/JLT/vySUuBhcvRS37Tni+/gffe2I79BakoTeIyD5bZKPefy7qZnXMbMdZtZvZv3d3eFeqUjw4jw7RICHnjw57wst7PH6JPQYc33ZmiUhzljt4nJdnbVnGMVF6D0CM7su7N8h8ab1BPHnarw+CT3GT958Vc1eQS1bblwWbGNC4jpZLCkQ99khEv14fZJ6iLm+LNYP9HhOs/VDAOsHemIf5MpcTx/9tySfAXA1gP0kD7hsj4RDq1fjL8oZXnFfP+Bla64Xd9+6wnexWKUOEnffuiL2M4UquZ419E0zu8zMLjCzf21mq122R8LhevWq1Fcer7+4YtXsBeeF8/WQ1B5iri+Lz7xr+byAWYkAPvOu5YnpCZRpaEhC53L1qjTnxYppkJNT06FcqSdh/YCfcsD0SgITxdXDSQsCgAKBRMDl6lVpXFRX6klYP1BLri+LI5vfjntuXTFnxlPShoMqJWYdgSRbtqvTMyeQlA9/GvjlbILO5SRh/UAjkjDjqVHqEUgkVG4i/vwSoY0kSJt1YcXc/K7OTOzWD6SNegQSCe2PG39+VTNr1ddvVvWaEgD41Wnv8gwSHQUCiUw7daXbkd/wXZC7atXKQ+hvwx0NDYk0KEmLoFoRxfBdkmcMtTP1CEQakIYyGVEM3y3SpIFYUiAQJ/JjhUTlC9IypBH28F27zBhqNwoEErkkXl1HNbWy3WnSQDwpEEjkknh1TXqXGw5hZmXb06SB+FGyWCKXxISh3wzKAGdWijijHoFETgnD9Epabigt1COQyCVxlbHfTlNJ2YEqDpJYfjotFAgkcknYorDalhuXIbNgbkIgs4CJ2YEqDpJafjoNFAjEiVxfFo9svBbrBnrwTy+8iNv2HMGVm+7HUH7cddM85fqyGH7n8jnBa/idyas771ISc0NpoRyBODOUH8fOkYnZ2zNms7fjWM5Xs13OjXJD8eV6q8phkk+SPErymyS7XLZHorX70Imm7nel3UtLeAnjnK95Xfe8DYrinhtKC9dDQw8CeKOZXQXg+wA2OW6PRCiKapfnKo0JzjDOOT9WwDcOF+ZsUEQAt7xJvaw4cL1n8QNmdrp0cwTAZS7bI9GKsv59q9KY4AzjnL1e0wA89OTJll9TguO6R1Dp9wH8rd+DJDeQHCU5evKk/njawdqVi5u634U0JjjDOOc0vo9JEnogIPltko95/Lup4pg7AZwGsMvvdcxsh5n1m1l/d3d32M2WCGzN9WL9QM9sD6CDxPqBnlglipO+v24r/M6ta2HrayZ+3We9RTu/j0kS+qwhM7uu1uMk3wfgdwC81SxGg8MSia253lh98Ve75nXd2DUyMWdsu90TnIOrl2Lw649iembux/EXL55GfqzQ9Jh+fqyAX750et79mQVs6/cxSVzPGroewMcA3Ghmp1y2RaRaWhOcub4sXnb+/GvE6TPWUp5g+MDxeUEFAF5+4Xlt/T4miescwecAXATgQZJHSH7ecXtEZqU5wfnC1LTn/a2U3fbLA0ye8v4dEj3Xs4Z+w8wWm9mK0r8PumyPSKU0Jzj9xu4JND2NNI15lqRx3SMQia00f4ENrl46b/EXUOwRNTs8pIVk8adAIOIjiVVSg5Lry8Jv5kYzPaK05lmSRoFAxEcSq6QGKRtAjyjNeZYkUdE5kRrSXGguiI3m05xnSRL1CETEUxA9ojTnWZJEgUBEfOX6shhcvRSLujrx7OQUhg8cb2rWUJrzLEmioSERD9pbt6hcibQ8PFSuRAqgofejfIzey3hTIJBYcvlFfK5ffu2kViXSRt+LNOdZkkJDQxI7rvcASGPpaT9K9qaDAoHEjusvYn35naVkbzooEEjs+H3htlLnphX68jtLyd50UCCQ2Amyzk0rVBLhrLQvqksLJYsldgZXL8Xte47MK3FQrnMT5peQSiLMp2Rv+1OPQGKnVp2bsIeHVBJB0kg9AomlbFen55d+eXgorCtUJYpra3Rar9ZhJIt6BBJLQZZBboYSxf4andbrevqvNM/1VpV/QvJoaXeyB0guctkeiY+gyiA3S7Nk/DU6rXfLvmNah5EwrnsEw2Z2lZmtAPAtAH/suD0SI0GUQW6WZsn4a2TYLD9WwKTPNpcaXosvpzkCM/uXipsvA3wvAiWFgiiD3ArNkvG2yCdvUxmY77rvWM3nSzy57hGA5CdIngCwDjV6BCQ3kBwlOXrypGZwpEHl1TkAdJCzQwwab46e17AZAPzyV6eRHysgP1bA8zU2pNfwWnzRLNyLcJLfBvBKj4fuNLO/qThuE4ALzWxzvdfs7++30dHRAFspcVZdBA4o9gw0ZBO9/FgBd913bN4XfmemAxect8B3WOjihRmM/fHbo2ii1EDysJn1V98feo/AzK4zszd6/PubqkN3Abgl7PZI8vglKbfs8x+GkHDk+rJYeP78EeWp6RnfIAAAm9csC7NZco5czxp6TcXNmwA86aotEl9+ScbJqWkNETnQbNK3qzOjnlvMuc4RbCf5GMmjAN4O4COO2yMxVCvJWCs5KeFoJunbmenAlhvVG4g7p4HAzG4pDRNdZWZrzEyXdzJPrSRjreSkhMMvaexFeZxkcN0jEKmr3hfJUH48opYIcHY2Vwe91n6fle3qVBBICNUakkTo6sz4JiN3HzqBrbnell53KD+O3YdOYMYMHSTWrlzc8mulSfkLvno2V5lWYyeLegSSCLXGmWdanAI9lB/HzpGJ2efPmGHnyIR6GA3yWucBaDV2EoW+jiAMWkeQTq/etB9nfP5c77l1RdNfPFds3O+5lL2DxNPbbmi+gSIx52wdgUhQ3rOyx/exZmcPDeXHfeuZtNrDEEkqBQJJjFpj98+fam5Nwe5DJ3wfq5cEFWk3CgSSKH4VSQE0tdK41lX/2pWLm2qTSNIpEEii1JqJMjk13XCit9ZVv2YNSdooEEii5Pqy6OrM+D6+a2SioSEiv6v+9QP+eQiRdqVAIIlTayppo1tZbs31Yv1Az2zPoIPE+oEe9QYklTR9VBKp7+MP+JaXIIB/3P6OaBskkgCaPiptZfOaZZ6b2wP+RdHyYwWs2n4QV2zcj1XbD6pyqUiJAoEkUq4vi3UDPfOCgV9pg/LmNoXJKRiAwuQUNu0dVzAQgWoNSYJtzfWi//JLMHzgOJ6dnMKirk4Mrl46Z4VxfqyA4QPHPffaLW97qVIIknYKBJJotTaa99rislqzm6yItCMNDUnb8trislozm6yItCsFAmlb9a72VSpZpCgWgYDkR0kayUtdt0XaR62rfZVKFjnLeSAguRjF/YonXLdF2ovXloqdmQ7cc+sKPLLxWgUBkRLngQDA3QA+BvhWBRZpSeXGKYR6ASJ+nM4aInkTgIKZPco6pX9JbgCwAQB6elQPRhpTa1aRiBSFHghIfhvAKz0euhPAHSgOC9VlZjsA7ACKJSYCa6CISMqFHgjM7Dqv+0n2ArgCQLk3cBmA75F8s5n9U9jtEhGRImdDQ2Y2DuAV5dskfwSg38x+5qpNIiJpFIdksYiIOBSbEhNmtsR1G0RE0iiR+xGQPAngx00+7VIAaRx2SuN5p/GcgXSedxrPGWj9vC83s+7qOxMZCFpBctRrQ4Z2l8bzTuM5A+k87zSeMxD8eStHICKScgoEIiIpl6ZAsMN1AxxJ43mn8ZyBdJ53Gs8ZCPi8U5MjEBERb2nqEYiIiAcFAhGRlGu7QEDyepLHST5FcqPH4xeQ3FN6/BDJJQ6aGbgGzvuPSD5O8ijJvyN5uYt2BqneOVccd0tp46PETzNs5JxJvqv0f32M5FeibmMYGvj77iH5EMmx0t/4DS7aGSSSXyL5U5KP+TxOkv+t9J4cJfmbLf8yM2ubfwA6ADwN4NUAzgfwKIA3VB3zhwA+X/r53QD2uG53ROd9DYCFpZ//IOnn3cg5l467CMDDAEZQrGXlvO0h/z+/BsAYgItLt1/hut0RnfcOAH9Q+vkNAH7kut0BnPe/AfCbAB7zefwGAH8LgAAGABxq9Xe1W4/gzQCeMrMfmtlLAL4K4KaqY24C8OXSz18H8FbW2wwh/uqet5k9ZGanSjdHUKz2mmSN/F8DwJ8A+BSAF6NsXEgaOecPAPhTM3seAMzspxG3MQyNnLcB+LXSz78O4NkI2xcKM3sYwHM1DrkJwF9Z0QiALpKvauV3tVsgyAI4UXH7mdJ9nseY2WkALwD4V5G0LjyNnHel96N4JZFkdc+51FVebGb7o2xYiBr5f34tgNeSfITkCMnrI2tdeBo57y0A1pN8BsD9AD4cTdOcavZz7ys2ReckGiTXA+gH8Fuu2xImkgsAfBbA+xw3JWrnoTg89BYUe30Pk+w1s0mXjYrAWgB/aWafIXk1gL8m+UYzO+O6YUnQbj2CAoDFFbcvK93neQzJ81DsRv5zJK0LTyPnDZLXobgz3I1m9quI2haWeud8EYA3AvhOaa+LAQD7Ep4wbuT/+RkA+8xs2sz+EcD3UQwMSdbIeb8fwNcAwMy+C+BCFAuztbOGPveNaLdA8A8AXkPyCpLno5gM3ld1zD4A/6708+8COGilzEuC1T1vkn0A/hzFINAO48Y1z9nMXjCzS81siRVLnI+geO6jbpobiEb+vvMo9gZA8lIUh4p+GGEbw9DIeU8AeCsAkHw9ioHgZKStjN4+AL9Xmj00AOAFM/tJKy/UVkNDZnaa5IcAHEBxpsGXzOwYyY8DGDWzfQC+iGK38SkUEzHvdtfiYDR43sMAXg7g3lJufMLMbnTW6HPU4Dm3lQbP+QCAt5N8HMAMgEEzS3SPt8Hz/iiAL5C8HcXE8fuSfoFHcjeKQf3SUu5jM4AMAJjZ51HMhdwA4CkApwD8+5Z/V8LfKxEROUftNjQkIiJNUiAQEUk5BQIRkZRTIBARSTkFAhGRlFMgEBFJOQUCEZGUUyAQCUCpFv7bSj9vJfnfXbdJpFFttbJYxKHNAD5O8hUA+gAkdtW2pI9WFosEhOTfo1jG4y1m9nPX7RFplIaGRAJAshfAqwC8pCAgSaNAIHKOSrtC7UJxx6hftMlmMJIiCgQi54DkQgB7AXzUzJ5AcWvMzW5bJdIc5QhERFJOPQIRkZRTIBARSTkFAhGRlFMgEBFJOQUCEZGUUyAQEUk5BQIRkZT7/5lZUm53vNndAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light", "tags": [] }, "output_type": "display_data" } ], "source": [ "%matplotlib inline\n", "\n", "d = 1\n", "n = 200\n", "X = torch.rand(n,d)\n", "y = 4 * torch.sin(np.pi * X) * torch.cos(6*np.pi*X**2)\n", "\n", "plt.scatter(X.numpy(), y.numpy())\n", "plt.title('plot of $f(x)$')\n", "plt.xlabel('$x$')\n", "plt.ylabel('$y$')\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "id": "JaUBFZGIJhxU" }, "source": [ "Here we define a simple two hidden layer neural network with Tanh activations. There are a few hyper parameters to play with to get a feel for how they change the results." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "vNjfSahuJhxU", "outputId": "e204d43c-dd0f-4dd8-d6a4-d1c42666f5d9" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "iter,\tloss\n", "0,\t3.96\n", "600,\t3.69\n", "1200,\t2.58\n", "1800,\t1.10\n", "2400,\t0.91\n", "3000,\t0.68\n", "3600,\t0.14\n", "4200,\t0.08\n", "4800,\t0.06\n", "5400,\t0.15\n" ] } ], "source": [ "# feel free to play with these parameters\n", "\n", "step_size = 0.05\n", "n_epochs = 6000\n", "n_hidden_1 = 32\n", "n_hidden_2 = 32\n", "d_out = 1\n", "\n", "neural_network = nn.Sequential(\n", " nn.Linear(d, n_hidden_1), \n", " nn.Tanh(),\n", " nn.Linear(n_hidden_1, n_hidden_2),\n", " nn.Tanh(),\n", " nn.Linear(n_hidden_2, d_out)\n", " )\n", "\n", "loss_func = nn.MSELoss()\n", "\n", "optim = torch.optim.SGD(neural_network.parameters(), lr=step_size)\n", "print('iter,\\tloss')\n", "for i in range(n_epochs):\n", " y_hat = neural_network(X)\n", " loss = loss_func(y_hat, y)\n", " optim.zero_grad()\n", " loss.backward()\n", " optim.step()\n", " \n", " if i % (n_epochs // 10) == 0:\n", " print('{},\\t{:.2f}'.format(i, loss.item()))\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "ic6YX9lNJhxU", "outputId": "036e3ef2-eba5-4b69-ad2b-1f4836ee4ccf" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEdCAYAAAABymAfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABKAElEQVR4nO3deXxU1fn48c9JCGTYEgJhyZAQ2YJChAgIilVZJKACKS6IWLVfrVZrW63GilrBpWqNWxd/ta6tFRQXOuIaQcEFAQUDhCXBEELChCUsYUuAkJzfH3duSCAh28zcOzPP+/XKi+TOnZlzgcxzzznPeY7SWiOEECJ0hVndACGEENaSQCCEECFOAoEQQoQ4CQRCCBHiJBAIIUSIk0AghBAhTgKBEEKEOAkEQoQIpdSlSqlLrW6HsB8lC8qECH5KqS7A554fL9Fa77GyPcJeJBAIEQKUUi8A/wPCgcla699Y3CRhIxIIhBAixMkcgRBChDgJBEIIEeIkEAhbUUoVKKXG+em9kpRSq5VSB5VSv6vnnFil1EKl1D6l1GtKqSeUUnc28vW/V0oN9Gqj63+vfyulHmvgnIC4FuF/raxugBDNpZQqAG7WWi9q5kvcCyzWWg85zTkzgZ+01pcopWKB1UDfRr7+08AjwBXNbJ+3BdO1CC+SHoEIZb2A9Q2cMw541/P9jcAnWuvyRr7+AmC0Uqp785rndcF0LcKLJBAIv/MM/8xUSm3wDFO8rpSKrOO8M5VSS5RSpUqp9UqpyTUe+y+QAHyolDqklLq3ic//EhgN/MPz/P4nPbe1Umo/kOx5j2xgIvDVSec9pZRy1fg5Qyn1hVKqtdb6CLAKSK3n7+E+pdRmz9DUBqXUz+v4e7pHKbVWKbVfKTXP/HtSSqUopX70PHcecMrfX1Ou5XTXAdDQtYgAp7WWL/ny6xdQAKwD4oEYYCnwWI3HxgERQB5wP9AaGAMcBJJOep1x9bxHY56/BGNoqb52ngXsrPFzCTD8pHM6A/uBFODXQDYQVePxvwHP1vP6VwFxGDdk04DDQI+Tru97zzkxwEbPe7QGtgJ3ea7zSqDC/DtszrU0dB0NXYt8BfaX9AiEVf6htS7SWu8F/gxMP+nxkUB74Emt9TGt9ZfAR3WcV5+WPh9gCLCmxs/RGMGkmjZW6D4H/AdjDP5SrfX+Gqcc9DzvFFrrd7XWxVrrKq31POAn4NyTTvub55y9wIeeNo3ECADPa60rtNbvAT+05FoacR2nvRYR2CQQCKsU1fh+K8Zdb01xQJHWuuqk85yNfP2WPh9O/fDcB3So47wsjGGXmVrropMe6wCU1vXiSqnrPVlLpUqpUmAQ0OWk03bU+L4MI7jFAW6tdc3VoFtPeyWNu5bTXQec5lpEYJNAIKwSX+P7BKD4pMeLgXilVNhJ57lr/Hy6ZfGNeX5DBlP7w3MtcPJcQjLwT4w76f+r4zXOPOk1zOf1Al4G7gA6a62jMYbLVCPatR1wKqVqnpvQwHNOey2NuA6o51pE4JNAIKzyG6VUT6VUDPAAMO+kx1dg3AHfq5SKUEpdDEwC3q5xzk6gdz2v35jnN+TkD89PgIvMH5RSTozhml8DtwPJnvcxH48EhgIL63jtdhiBrMRz7i8xegSNsQw4DvzOc21TOXVIqdHX0tB1NOJaRICTQCCsMhejGmY+sBmotRhKa30M44N7IrAb+H/A9VrrnBqnPQE86BlauacZz6+XJ02yE1Dz/DeAS5VSDqVUR4wP02e11gu01mVABsZ8h2kSsERrfXJvB631BuAZjA/1nRhDMksb0zbPtU3FSAHdizHRPL+Z1xLViOs47bWIwCdF54TfeWEhmGWUUo8Du7TWzzfi3BXATVrrdT5vWDME07WIlpFAIPwukAOBEMFIhoaEECLESY9ACCFCnPQIhBAixAVk9dEuXbroxMREq5shhBABZdWqVbu11rEnHw/IQJCYmMjKlSutboYQQgQUpVSdK9BlaEgIIUKcBAIhhAhxtgkESqlwpVSWUuojq9sihBChxDaBAPg9Rr11IYQQfmSLQKCU6glcBrxidVuEECLU2CVr6HmMjcTrqvUOgFLqFuAWgISEhiruChE8Zry8jKWb91b/PKpPDHN+dZ6FLRLBxvIegVLqcozCV6tOd57W+iWt9TCt9bDY2FPSYIUIOg+6skm87+NaQQBg6ea9JN73MUMe/hxXVlO2VxCibpYHAmAUMNlTiOxtYIxS6k1rmySEtR50ZfPm8sLTnlNaXkH6u2skGIgWszwQaK1naq17aq0TgWuAL7XW11ncLCEs9daKunaKPFVFlSYjM9fHrRHBzvJAIIQ4VWUTikEWl5b7sCUiFNhlshgArfUSYInFzRDCUk0d6omLdvioJSJU2CoQCBHqHnRlM+ekuYGYsv10OHqYrZ3iTjk/IkyRnprkr+aJICVDQ0LYhCvLzZzlhdQcFGp7rJx35/yRha/czg1ZH0ONIaNoRwQZVw0mLcXp/8aKoCI9AiFsIiMzl5NnBh5Z+CJn7HWzynkmD3/+Tx7udghefBHatrWkjSI4SY9ACJs4edJ36rovuHLdF/z9/Gu46/a/wiOPwJtvwnnnwebNFrVSBCPpEQhhE3HRDtyeYNB7zzYe/fyfrIgfxN9HXcPTE8+ElHEwfDjMmAFDh8J//4ur5zlkZOZSXFpOXLSD9NQkGSoSTSY9AiFsIj01CUdEOG2OH+OFD57kaKvW/H7SPVxz/hknPtwnTIBVq6BPH5g8maI778NdWo4G3KXlzJyfLQvMRJNJj0AImzA/7Ctvv50zSwq454Y/c9/N4069w09MhKVL+Xzk5fz26zdx9R/F5s7xAJRXVJKRmSu9AtEk0iMQwkbS8pdzxfIF8Ic/8PS/76//Az0yklnnGwvwL/lpRa2H3LLATDSRBAIhbMCV5ebKe+dwYMYNbOiZxAfT7mjwObuiurK2e18u+Wl5rePhSvmqmSJISSAQwmKuLDcz52dz1ztPgdbcelk6932Y2+BYf6XWLOw7gpTiXGIP7at1XIimkEAghMUyMnNpfaCU87Zm8+rwNIqiu1eP9Z+OM9rBwn4jCUMzNm9FreNCNIUEAiEsVlxazsiibMLQLE0cXOv46aSnJrE1rg+FUd24xBMIHBHhUnJCNJlkDQlhsbhoByMLsymLaMOaHv1rHT8dcyJ52ZILSFu+gL4OuGNysmQMiSaTHoEQFktPTWJUYTYrnWdRER4BNP7OPi3FybTH7qBNZQWLBh2RICCaRQKBEBZL69ma/iUFbOyfgsIY439iahPu7C+4AGJi4IMPfNpOEbxkaEgIq331FQC3zr6ZW89rxqb0rVrB5ZfDhx9CRQVERHi5gSLYSY9ACAu5sty8/8x/ORwRyYVfHmx+eYgpU2DfPvj2W+82UIQEywOBUipSKfW9UmqNUmq9Uuphq9skhD+Y6wfO/imLH3oOpPBgRfNrBaWmQmSkDA+JZrE8EABHgTFa68HAEGCCUmqktU0SwvcyMnNpv283/fYUsaxXMkCj1g/UqV07GDcOXK5am9cI0RiWBwJtOOT5McLzJf+TRdArLi1nZOFaAJYlnF3reLNMmQJbt8Latd5ongghlgcCAKVUuFJqNbALWKi1XlHHObcopVYqpVaWlJT4vY1CeFtctIPzCrM50Lot67v1qXW8WSZNAqVkeEg0mS0Cgda6Ums9BOgJnKuUGlTHOS9prYdprYfFxsb6vY1CeFt6ahLnF2XzffxAKsPCgRauDO7Wzdi9zOXyXiNFSLBFIDBprUuBxcAEi5sihM+ldYXEvW42JA1t3vqBukyZAllZUFjotXaK4Gd5IFBKxSqloj3fO4BLgBxLGyWEPyxeDMDvHruFLU9extL7xrR8ZfCUKcafCxa0sHEilFgeCIAewGKl1FrgB4w5go8sbpMQvrd4MXTqBIMHN3xuYyUlwYABMjwkmsTylcVa67VAitXtEMLvliyBiy6CMC/fj02ZAs88A6WlEB3t3dcWQckOPQIhQk9hIeTnw8UXe/+1L70Ujh+XVcai0SQQCGEFz/wAo0d7/7UHeZLucpuxME2EJAkEQlhh8WLo3PnEh7Y3xcRAly6waZP3X1sEJQkEQvjZg/9bi/t/n/BJ5yT6PPAZD7qyvf8m/ftLj0A0mgQCIfzoQVc2X2X+gPNACct6nU2l1ry5vND7wSApSQKBaDQJBEL40Vsrijhv66n1hd5aUeTdN0pKgh074MAB776uCEoSCITwo0qtOa9wLSXtosnrHF/ruFclecpUyDyBaAQJBEL4UThwXuFalscnGwXizOM1vveK/v2NP2V4SDSCBAIh/OjXfSLofmgv38cPrHV8+oj4ep7RPAsORlKpwvjb//uIUU9+2fydz0RIkEAghB+l9zL+zItNBIyewHUjE3gsLdlr7+HKcnP3BzkURXWj91437tJy0t9dI8FA1MvyEhNChJQco57iW8/eaJSN9oHZC9ZTUaXJj3HSe+82ACqqNLMXrG95UTsRlCQQCOFPubkQFQVdu/rsLUrLKwDYEuNkZFE2SlehVVj18WDjynLz8Ifr2VdmXF+0I4LZkwdK0GsCCQRC+FNOjlEd1NuTw3XIj3HStuIo3Q/uYXvH4NvMyZXlZvaC9acEuNLyCu6ct5qVW/d6dcgtmMkcgRD+ZAYCH+rUNgKA/JieAPTe6651PBi4stzMnJ992l7OnOWFMi/SSBIIhPCXgwehuNjngWDWpIFEhCs2xxhDI733biMiXDFr0sAGnhk4Zs5fS3lF5WnP0RjzJaJhEgiE8Bczpz+pmXsSN1JaipOMKwdT0bUbh1o76L3XTbvWwTMK/KArm/KKqkadW1pewYyXl/m4RYHP8kCglIpXSi1WSm1QSq1XSv3e6jYJ4ROejCFf9whMh45WsqVTHL33uiktrwiaFNK5K5q2H/PSzXt9U9gviFgeCIDjwN1a67OAkcBvlFJnWdwmIbwud8kPHA8Lo/9ruT5f5HUihbRn9RyBmUIayFxZbqisZGThWpK3/0SPAyW0Pt5wNpTXazkFGcv7i1rr7cB2z/cHlVIbASewwdKGCeFFriw3kd/9SKvoHhwLj8BdWs7M+cZdqi/SHM1J1PwYJ5M2fk2biqMcjWgT8CmkGZm5TNr4NX/96Jlaxw+0acfutlHs7NCZWeNuZZNnwZ7J67WcgozlgaAmpVQixv7FK+p47BbgFoCEhAT/NkyIFsrIzOXVkiLyY0586JdXVJKRmevTfPctMU7C0PQq3X7Kh2MgKi4t52cFq9nr6Ej6pb+ny+FSOpftJ/bwPgaoMoas/pYbV33E/RPuqPU8r9dyCjJ2GBoCQCnVHngfuFNrfUrtXK31S1rrYVrrYbGxwZcTLYLbjr2HOGOfm82elE5TcWm5T97PTBU9kTl0YhgqkOcJ4qIiGVm4luXxg/ii7wjmDU7l/513NU9NvI3zfljEqqEXMTF3KRGVtXs+3q7lFGxsEQiUUhEYQWCO1nq+1e0RwtuG6gO0qTzO5s61A0FctMMn72emim6pIxBkZAZuRdJZgxz09GzqY3JEhPPEVOPnC/70OzodOchFBasB39RyCkaWDw0ppRTwKrBRa/2s1e0RwhcujzQ6uZtjTtyZOiLCSU/1TSppWoqTO+etpqy1gx3tY+jjqTkEvuuF+MP4ncbUYf7A4SiMQJqemnRieG38eOjcmVciNsGTsy1rZ6CxPBAAo4BfANlKqdWeY/drrT+xrkn2V1d9lYFxHViev49KrQlXiukj4uVOyAZcWW6Kl68GIL+z8YGlgCuGOn06P+CMduAuLSc/pidn1OgR+KoX4heLF0O3bsx59pd1l+mIiICrroI33oDDh6FdO/+3MQBZHgi01t9i/F6I03Blubl//lrK6llIU1pewdLNe6t/NvfCfXN57Zxr6Sb73+wF67m3pJA9jo6UOjoCxqrXxTklPn3f9NQkZs7PJj/GyeU534DWOFq38lkvxOe0NgLB6NGnr9V07bXw4ouwYAFMn+6/9gUwywOBOL2GAkBTvbm8kHnfF5Jx1RCpzugHrixjMVefPdtOmR/w9RCN+e/70w8JRK8+REz5AXS7Lj59T5/atAm2bzcCwemMGgU9e8LcuRIIGskWk8XiVK4sN2f+6VNjnLexQUBr46sBFVVw57zVstrSD8yJ2d573dVF4Ez+GqLJ6xTnacM29pVVMHN+dmBmDi1ebPzZUCAICzMCwGefwZ49vm9XEJAegQ25stykv7uGiqqTPtS15qxdWxiXt4JepduJKj9Ip/KDRB85RHT5AaKOHKIsIpI1PfqTFZdElnMAq3v0Z1/bqDrfxxw2kqEi3ykuLSeq/CCxZaWnpI76Y4gmIzOX8I5GIDhjr5uVPQf6Zf2CTyxeDE4n9O3b8LnXXgsZGfDee3Drrb5vW4CTQGBDGZm51UFA6SrOcecwYdN3pG5aRsL+nVShKO7YhVJHR0oj27O9Yyylke3Z5+hIp/IDDNm+iduXv0srbfQktnTqwbKEs/n7+decUpd+zvJChvWKCbwPhQARF+2gm3sjQK2hoU5tI/zyd15cWo6K6srR8Fb0qTFhHGiZQ64ft3HRx5+zJDGFp/+yuHamUF0GDzZqOr31lgSCRpBAYAOuLDd3v7OayhodgLgDu/j18veZuGkpsYdLORbWiqWJg3nhvKtZ1HcEe9pF13qNk7OGHMeOkLwzjyHFuaQU5zJ1/WLSNizhhZFX88q5P+doq9aAMWl557zVzF6wXnZ18oH01CS+X/oRcCIQOCLC/VYSOs6TObQ1Oq7WWoJAyhxyZbl57V8fkXa4lO8Szm5ceQ6ljF7BrFlQVATxsqDsdCQQWMyV5ebOeaurf44uP8Dty97lhh+ND4+FfUfwef/zWNxnGAfbnEiFU8CMBjKAHnRl89LyQQA49+/igS9fIf2b/3JV9iIeGfsrvux7bvW5ZnVK8E3tm1CVluLkrC5HqQhvhTuqG86T8959zMwc2hITR589xloCX65f8IWMzFzG5a8GqF5I1qjhrenT4aGHYN48uOceP7Q0cEkgsJg5mRhZcYRfrvqQ25a/R/ujZbyXPJbnLphxylCOIyKMJ6ae3agPksfSknksLbk68+j2n9/PqILVPLzwRV57/xG+7D2MR8b+igLP6tOKKh2YY8c213+fG5L6k/fUZL+/t/lvueubREZvXklCh9b84dKzAurf2F1aznmFaymK6sa2qG7Vxxsc3urbF84918gekkBwWhIILOLKcpORmcv2vYe4OvsL7vp2Dj0O7WFRn+E8ddENpxQI69Q2glmTmjd0k5ZiLFx60JXNHGDi//2dG1Z9yO+XvkXma7/htrT7q3sHgTZ2HBBycmCgdbuDpaU44Ybx8NXbfD3tDOgbOEHAleVG6SpGFK5jYb8RtR5r1PDW9Olw111+2SI0kEn6qAUedGVz17zVHNtWzBvvPMRTn/2N7R27cPW1T3LzlbNqBYGCJy+j4MnLyHpofIvv4h5LS+a5aUOoatWaV86dyphf/Yvc2ERe+OAvDN1mLN0PpLHjgFBRAZs3W/8hZO6KlhtYdYZmL1jPmbsK6HTkIMsSzq71WKOGt6ZNM+YL3nrLRy0MDhII/MyV5WbO8kJGbcnik9d/y1B3Dn+c8FumXvc038cPqnVuv67eXx6fluLkmasHExGuKGkfw41XPcz2Dp157b2HGVBSQNmx45xx38c+3zglZOTnw/HjPt+eskH9+xt/BlggKC2v4LytxtzVyYGgUTdGPXrAmDHG8JDsSVAvCQR+5Mpyc+/bP3L312/wxjsPsc/RkcnXP8u8wamnLJnv17UdC/9wsU/aYe5p26ltBHvbRnH9tEc5GtGGf78zi7bb3WiozsyQYNBCft6esl5dukBMTMAFAoDzCteS3ymOHR2buSr62mshLw9WrvRuw4KIzBH4iSvLzV9f/5I3//ck527bwNtnj2f2uFs4EhFZfY4z2sHS+8b4pT3mvIHpF22q+MeLd/LGOw9x5Yy/sK9tVOAuPLITMxBY3SMw27Bpk9WtaDRXlpvwqkrOLVrPR2deWOsxc7+FRpk6FW67zcgeGj7cy60MDhIIfMycFE5cvYz5H/yFiKrj/G7SPSw46+Ja5yn8s9K0Pt9G9uCmKx/izXl/4vX3ZnPtNY9T1tohk8ctlZtrDE9E1b2626+SkiAz0+pWNFpGZi4Dd26m47EyliXUTpNu0jqM6GgYNYrSjz/nsi5fUlxafmr56hAnQ0M+ZE4KpyzL5PV3Z7OjQ2cuv+H5OoPAjJEJlv6njIt2sLLnQO6Y/EcG7djMi/97nIjKCsKUkuGhlsjJsUdvAIx5gu3b4cApGwDaUrEnbRRgeXPmB2rI6TeEDrnrObBztwx91kECgY9c8uwS3lxeyIwfP+ZvCzLIcg7g6hl/qc7ZN4UrxXPThlhe7yc9NQlHRDiL+o1g5oTfcmFBFnd+O5dKreUXprm0tlfaohmQAmR4KC7awXlbs/mpczwl7TtVH3c2I7PthSon4bqqOjsOTixKExIIfOKSZ5fw085D/P7buTy28J980fdcrr/q4Vorg8FY4fnM1YNt0T1NS3HyxNRkwpXi3bMv4f2Bo7n5BxfO/bsor6hk9oL1Vjcx8JSUwL59Egia6d4xvRm+bX2tbKHmrope1DGRY2GtGFFU+/+xDH0aJBB42YOubPJ2HmT2on9x19K5vJM8jl///H6ORrSpdZ4z2sETU5NtEQRMaSlOqjwpdk9feD1VKox7v/oPYKTxSdnqJjIzdOwyNNSnj5Gd9tNPVrekUaYcL6ZdxRFyzxqGomW/MzGxnVjbox8jimr/H5Z1MwZbBAKl1GtKqV1KqXVWt6UlHnRlM2/pZv764dPc+ONHvHjuVO6d+Hsqw8Krz1HA89OGsPS+MbYKAibzF2N7x1heHp7GlI1fkeI2Ml/mLC+UIaKmsEvqqCkyErp3h61brW5Jo6x/60MAPo3p3+LJ3fTUJH7slUzyjjwcx44AgVdzyZdsEQiAfwMTrG5ES5hB4MX/Pc7kjV/z+MW/5MnR/3fK+gCrJ4UbUvMX48WRV7KrXSce/PIV0BoNMqbaFDk5xodvQoLVLTmhV6+ACASuLDebl3zPto6x7G0b1eLJ3bQUJ4OmTyKiqpKhxTm27JFbyRaBQGv9NbC3wRNtasbLy3jruy08/+HTjN38AzNT7+ClEVeccl6/ru0snxRuSFqKszpHu6y1g6d/9guGFudwae5SQMZUm8TMGAqzxa+ZoVcvKCiwuhUNysjMJW7fdoqiu1cfa+nk7vnXT4awMN7sf9S2PXKr2Oh/6OkppW5RSq1USq0sKfHtpt+N5cpyM/Chz/gubzdPfvp3LstdyqNjbuatIad2bny5UtjbZk0aiNmPeS95LBtjE7lvyeu0OX5M0kmbIjfXPvMDpl69jPr8Vd7ZA9tXikvLid+/k8Ko7qccb7aOHSElBb7+uoWtCz4BEwi01i9prYdprYfFxsY2/AQfc2W5mTk/m8NHj/PQFy9z1bpFPDfqWl4dnnbKudeNTAiYIABGr2DGyAQUUBUWzmNjbiZh/05uWPWhpJM21pEjsGWLfeYHTL16GYXwtm+3uiWnldgujG6H9lIU3a3W8RZP7l50ESxfbvz7iGoBEwjs5uEP11NeUck93/yXX676kJeHp/HXUdNPOe+6BjaPsSuzUmm4UixNHMKiPsO547t5xJTtl/zrxsjLM+667RYIEhONP20+T/DAWcYHflGN/Qe8Mrl74YVw9Cj88EPLXifISCBohgdd2ewrq+C25e9yx7J3mDs4lT+PvumUieFADQKmmumkT4z+P9pWHOHOb+cCMlfQIDNX345DQ2D7QNDOXQhQPUfQqW2EdyZ3f/YzADa8/RGjnvxSKu162CIQKKXeApYBSUqpbUqpm6xuU30edGXz5vJCrvvxY/741X/44MyLeHD87acEgVF9YgI6CJjMrvjmzvHMSZnItas/pc/uIsm/bkhenvFn377WtuNkARAIXFluFn2yHIAizxzBkQovzWnExLC/7wD2fboQd2m5lJvwsEUg0FpP11r30FpHaK17aq1ftbpNdTH3EpiQu5RHFr7Iwr7ncvdld1F10jqB60YmMOdX51nXUC8yS08APD/qWspaO7jnu7mSf92QvDzo2tWYoLST9u2NctQ2DgQZmbn02FNMeas2lLSLBrxbDmJRlySGbNtIq8rj1cdCfbhTqo82kivLzd3vrCHFvZHnP3qG1XH9+e3kezkefuKvMNoRwepZ4y1spfeZXfGMzFyKgY+HT2Ta8g8I69na2obZXV6e/XoDJpuvJTAzhoqiutXqaXtrOPLLbmdyRcUHDNq5mdVxJ25oQnm40xY9ArszM4R67nXz8vuPsqN9Z26+4qFaewkoYPZk6/al9aW0FCdL7xvDlicvo+vvfk3Y8QoeueYBGVs9HQkEzRYX7SChdAeF3s4Y8igcOBSAc4tqFzII5eFOCQSNkJGZS+T+vfz73Vko4JdXzWZv29r15e2+YtgbXFlu7lhbweoe/bkqeyHufWUhP7Zap/JyI1ffpoFgs6MzZXn5nPHHj2wZzNPH9zd6BDUWk3mzHMRNV57HlhhnrUAQ6uUmJBCchivLTcojn7O7pJSX5z9G3IHd3Dz1T2ypUUranBMIhonhhmRk5lJeUck7Z1/CmSUFJO/Ik8qkddmyxfjThoHAleXm3d3htD12hKgjB205UZqWEEn7Y+Uc6B7f4mJzdb5+ihN10UWMcG8gvKpSyk0ggaBeD7qyuXPeakoPH+W5j57hHHcOd15+Nz/2PLP6HLvsJeAv5hjqh2deSHmrNkxb+zlgVCa10weJ5eyaMYQRzLe0M/b+de7fBdhwotQTSH//q/FsefIyn5SDSJw6kQ5HDrP52gQpN4EEgjqZ2UEAMxe/zqWbvuPx0b/k0wEXVJ9jp70E/MUcQz3Yph2fJJ3P5A1fEVlhrNCUXkENNg4ExaXluKO6AtDzwK5ax23D7FH17u2797jQ2AP5uT+9ImsJkEBwCjM7SAPXZX3CLT/8j/+ccxmvDP95rfNCsStZcwz13bMvoeOxMlI3LQOMXoHwrDN58wv2RXagz1PLbLeHQ1y0A3dHo0SLc39JreO2kZ9v/HnGGT57C9eecLZFdWPApixZS4AEglrM7KBKrbkwfxWzF77IF32G8/DYW2qlsTmjHSEXBKD2PrEr4gexNbo7V2cvtLBF9mIuNkzYt52tnXpQqTVvLi+0VTBIT03iSMdOlEW0wenpEdhuonTLFoiNNdY8+EhGZi4r4gcaE8ae1fO2GyLzIwkEHmZPoLyikn4lW/nHB3/hpy4J/G5Seq0FYxFhyl6/NH5mlqjWKox3k8cxautaepbuqD4eyt5aUQRA4r5iCjr1OOW4HaSlOHniirPZ2akbzgO77DlRmp/v094AGENhK3oOonP5AfruKap1PBRJIMAIAunvrqFSazofLuW19x/hSEQbbrryIQ63aVt9XtuIMDKuCq15gZPNmjSQiHCjd/T+oLFUobgq+wu0JmS71aZKrYmorMB5oISt0XG1jttJWoqTM4YOZEK7I/acKN2yxbfzAxhDYSsSBgEwokYaqa2GyPwo5AOBmR1UUaVpc/wYL81/jC6HS7n5ij9R3NGYVAtXiuenDWHDoxPt90vjZ2kpTjKuHEynthFs7xjLN2ekcGX2Ig4cPhLSY6xg/D/puX8X4bqqVo8g/KQ6VLZg10VllZVGu3zcI0hPTWJXbE92to/hXM+G9rYbIvOjkA4EM15expue7CC05qlP/srQ4hz+cNldrO3RHwjN7KCGpKU4advaKK0x7+zxOA+WMGrrmpAeYwWYPiKeXvuKAdhaIxBMHxFvVZPq16sX7N0Lhw5Z3ZLatm2D48d93iMwh8hW903hZwVZJHTwUnXTABWytYZcWW6Wbj6xO+adS+cyZeNX/OWiG2qliYbyf47TMcdSF/Udwb7IDly9diHfnHEO7hAdYwVjD4ePPpoDQEGnOMKVYvqIeHuuM6lZhXSgjUqj+CFjyJSW4oSHfgNTp/L1iDCwwe+5K8tNRmYu7tJywpWiUmuc0Q7SU5N8+jkUsj2C++evrf4+bf1i7lz6Fu8kj+OfI66sPt6pbYQEgXqYY6nHWkXgGngx439aRlT5QRShPVdwedsy6NiRH/86nc1PXGrPIAC2LEftynLz+AufAHBl5g7//D+aMAHatYP33vP9ezXAzFo0b6bMuSV/pLaGZCBwZbkp89Q3H1GYzVOf/JVlCck8kPqbWmmisybZ6E7JZtJTk6r3NX7n7EtoU3mcKRuWoAnxxWVmsTk7zgvUZLNAYH4Idigu4rgKI4sO/plzcjjgsstg/nxjSMqPXFnuWpvjmLse1sXXw64hFwjMNFGAPnuKeGn+YxRGd+fWnz9ARfiJFMjrQqCIXEukpTgxc2E2du1Ndrc+XJW9CAjxkhN5edCnj9WtaFiPHhARYZtAYNaxit+/g+KOsVSGhftvzumqq6CkBL75xvfv5eHKcpP+3ppam+PsKzv9okx3abnPVkCHVCCouWCs8+FSXn93NsfCI7jxqtkciDyxeOX5EKof1BLOGql2H555Ick7N9PjgLFaNSQnjY8fN1IfbVha4hRhYRAfb5tAYM45JZTuqLVhvV/y+idO5HhkJO/f/7zfyk08/OF6Kiqbnlbsq2EiWwQCpdQEpVSuUipPKXWfr97HvOuIrDjCK+8/SqwnTXRbjXK3z08bIj2BRqqZavdFn3MBGLPZ2BQ8JBfmFBYawSAQAgHYKoXUnHOK37+Twqjupxz3JdemUhadMYwLs79GVVX6fEzeleVu8O7/dHzRU2owECilFiqlBnv1XWu/fjjwAjAROAuYrpQ6yxfvVVxajtJVPPfRswzevok7J93NGs8ORWY5aQkCjZeW4qxeUby5c0+2RndntCcQhOTCHBsXm6uTjQJBemoSMbqC2MOl1fsQ+CuvPyMzl4/6nU/s4VKGuTcCvhuTN0clTsfsaZ9u/Ym3b7Qa0yP4I/C8Uup1pVSPBs9uunOBPK11vtb6GPA2MMUH70NctIOZi19n4qbv+POYm8jsfz4QeuWkvWnWpIHGnsZK8WWf4YzaupZojofmwpxADATbt8OxY1a3xFioONzY33lbVDe/lr4oLi3nyz7DOdKqNRNzl9Y67m3mqER9oh0RLL1vDAVPXsbmJy6tNfxak7dvtBoMBFrrH7XWo4GPgM+UUrOUUt5shROoWYxlm+dYLUqpW5RSK5VSK0tKSk5+uFH+3/7l1dVEXx1mxBpZMNYyaSlOnpiajDPawZd9huM4fpQhm7PIyMwNvQnjvDwjC6WHL+6XfKBXL6PgWpE9aiGNbXMYgL89cIVfS1/ERTsoa+1gSe+hTMxditJV1ce97XTBJSJMnbLdbXpqknGjVYMvekqNmiNQSikgF/gn8FvgJ6XUL7zakgZorV/SWg/TWg+LjY1t1msM7tOV4tGpvDr19yil7FlwKwClpThJT01iTe/BHI6IZOzmH0KzrG+gpI6abJZCWr0PgR8Wk9Vkfth+kjSK7of2co47x2fDUvUFl3Cl6qxjVvNGyxe7tZkaXFmslFoKnAGsB5YDNwI5wO+VUj/TWt/Swja4gZpr8Ht6jnnfzTcTd9NNfB0ov6gBJCMzlwM6nG8ThzAm7wf+dImuHmcNmUCblwcDBljdikb7/FAbxgPpTy/gu+/x+erVBuXnG4u7mnmj11zmNb8QXsHRT/7K1QUraHP3tT75u0hPTWLm/Oxaw0OOiPDTfrinpThPPLZ3L8TEeL1djekR3AI4tdaXaK3/pLX+SGudp7X+LfAzL7ThB6CfUuoMpVRr4BpggRdet24SBHzC7PJ+2Wc4zoMlJO3eWut40KushM2bA2Z+wJXl5p7le6lC4Tywyx49uC1bjN6ABb+jaSlOFs6eRJvLJjJt6wrSBvtmeK/Zd/iHDsFDDxkpvz5Y79Bgj0Brfbplope1tAFa6+NKqTuATCAceK2B9xQ2FBftwF1azuLewwAYm/c9ubGJoZM95HYbk64BEggyMnM5UBXGzvYx1TuVWd6Dy8/3ebG5Bl15JSxYAN9/DyNH+uQtat3hN6SyEl5/Hf70J9ixA665BhISvN6mFq0j0Frne6MRWutPtNb9tdZ9tNZ/9sZrCv8yx1l3dejM2u59Gb15ZWiV9Q2wjCGzp+aO6krPAztPOe53Wp/oEVhp8mRjxbUNag/x+eeQkgK/+pXx97JsGbz11om5HS+yxYIyEfhqdnkX9x7OOcU5PDO6CXc+gS7AAoHZU3N37Ipz/65TjvtdSQkcPmx9jyAqCsaPNwKBVRsK5ebCpZdCaqoxJPTOO7B0qc96KCCBQHhRWoqTpfeNYfBtMwjXVSx87j9+Wa5vC3l50KYN9OxpdUsaxezBuaNi6XFwN2FVldb24CzKGKrTlVcamVQrV3rl5U4uLlfv78ORIzBrFpx9Nnz3HTz9NGzcaNRC8vG8ScjuRyB8w5Xl5v4cxVftohm9+Qf+N3B09UrKoO4d5OUZd7NhgXFvZf5b/LSuJxFVlZwdXs6NU4daOz8A1vcIAKZMgVatjF7B8OEteilzJbGZJWROysNJvw9ffAG33QY//QTXXgvPPgvdutX1kj4RGP9rRcDIyMyl7Lhmce9hXJS/ivCqytDYucxcQxBA0lKcjBp7DgCtigqtXQRo9ggSE615/5o6dYJx44whmRauuq5rJXGt34ddu+AXvzDeT2tjXmDOHL8GAZBAILzMnGz8os+5RB09zFBP7ZagTiPVOiADgSvLzePrygCIszqFND/f+PBr187/712X22+HggK4445mzxU86Mqud8e+7fsOw8svG+tO5s0zsoLWroVLLmlBo5tPAoHwKnOy8dvEIRwLa8WYvO9rHQ9K27dDeXnABYKMzFw2t+0CQE/PhLFlvTc7ZAzVNGkS3H+/8WH9t781+ekPurJP7Id+kqSSAlzzZsIttxjzAWvWwCOPGOVJLCKBQHiVOQl5uE1bVsQPYuzmH4I/jTTAMoZMxaXllLeOZK+jI84Du2od97stW+wxP1DTo4/Cz38Of/gDfPZZk546Z8WpQcBx7Aj3LX6Nj1//HUn7t8N//gOLF8OZZ3qrxc0mgUB4Va000j7D6beniL+e2zH4J4oh4ALBiRTS2OpFZTWP+83x48ZeDnbqEYAx8f/GG5CcDNOmQU5Oo5968mjS2LwVLHz1Nn79/Xy2pU2jTd4muP5621Q6kEAgvM5MI33oX/cCML5glcUt8rG8PCPLxAcrPn3pRApp1+oegSW9t6IiYwWt3XoEAO3bGyuNIyON4aK9e5v09KHbNvDau7N59f1HOdzawZUz/kLi/LnQubOPGtw8kj4qfKdvX+jfHz7+GH77W6tb4zt5ecbdbKvA+nUye2n7F8fh3PIjzqhI0icM8H/vzUwdtVuPwJSQAC4XXHyxkdP/2WfG6uN6uFYVkbrpO25ZMZ+hxTnsdXTkiYtv5NVhabSObO23ZjdFYP3PFYHn8svhH/8wVki2b9/w+YEoADOGTGkpTrJHJtN26fuUb99JRqaqPu43ZuqozXoEriw3GZm5FJeWExft4G8PPsXQh+40Mop+9zsjw8n8atvWSDV94w0Gz3yEtL1uCqO68dC4W3k3+RLKW0cC8PTUs629qHrI0JDwrQkTjF8QH1RMtIUATR01ubLc/KvQyHOPO1BiTQppfr7Rm7LRqmxzIZi7tByNsRDsumNJbLrxdnjlFSPbp08f6N4dOnSA8HDjz1tv5WBrB3dMvpfRt7zEG0MnVQcBsO+iSukRCN+64AIqW7fhncdf4/6vqoiLdlhf996bSkrg4MGADQQZmblEtzNSSJ37d7Gue1//VyHdssUopBYe3vC5flLfQrBf9p/K0iVXGwvBDh+u/XXkCNcVRvNtQnKdk8D1bTtpBxIIhE+5cvbSLW4AQ3JXokfdWP8S+0AVoBlDpuLScg517ApAzxoppPUthPIJszyHjdSXQlu8/whcVH/1/W/v+7jex+ycQi1DQ8KnMjJz+SZhMGeWFND5cClg4aIlXwjwQBAX7WB/ZHsOtXbUqkKqwOfDQ64sN6Oe+IJDa9fz7qH2tipOWF8KbZhSp21neD3poAp73/hIIBA+VVxazreJQwA4f+uaWseDwqZNxpCGHWrkNEN6ahJKKbZ17Er8/h3VxzX4NFibY/BVRUW0P1bOmg49rN8hrYa6No0HqNSa9PfWnNJOs8JoZT3lKGaMtHdqsaWBQCl1lVJqvVKqSik1zMq2CN+Ii3awrlsf9rdpx6gagSBoSk6sW2ekyLa2Z1pgQ9JSnGggr3M8/XfXXg3ry2BtjsH33VMEQF7nnrbqKZoLI8PquMGvqNQ8/OGJTRRdWW7S31tT53BauFJcNzKBx9KSfdncFrO6R7AOmAp8bXE7hI+kpybRpk1rvus1mAsKskDr4Co5sXatkUESwJzRDnJje9GrdAdtj534MPNlsDaDzIlAEF/ruB2kpTipqqfe3L6yiurv75+/lorKU0/s1DaCzU9cavsgABZPFmutNwIomyyzFt5njouuyx7OxE3fcW7lPq6dOsbW46WNdvCgkfFy001Wt6RF0lOTyFxpLObqv7uQ1XFJRIQpnwZrc4/rvnuKKI1sz+620dXHA4U5PFRWUVXn4zWDhd1Z3SNoNKXULUqplUqplSUlJQ0/QdhGWoqT9KduB+CdvoeDIwiAMSwEAd8jANjUNREwKmMCxuymD5lj8H33bDN6A0oFXE/xD++s5q53VlvdDK/weSBQSi1SSq2r42tKU15Ha/2S1nqY1npYbGysr5orfKVfP8q6x7H4hbkNb9kXKNauNf4M8ECQkZlLfoeuHI6IZIAnEFRUap+O15tj8P33bmNz53ic0Q6emJpsu5uETm3rLyVRpU+/VUG0o/7n2o3Ph4a01uN8/R7C/lyri6nsNpCxuctQVZXBsZ4gOxs6dgy4YnMnKy4tR6swfuqSQFLJ1lrHfSktIRIOlzLt+vFMu3uMT9+ruWZNGsid81Y367mzJw/0bmN8KGCGhkRgy8jM5auEwUQfOcTAnUaRMTtliTTL2rVGieIAn+Myx+VzYhONoSHPba4vx+tdWW5ue+BNAO7JPmbb3mFaipPrRiY0aaRMAdeNTAioGxyr00d/rpTaBpwHfKyUyrSyPcJ3ikvL+a7XYAAu2Lq6+rhfV7B6k9YnAkGAM8frc2N70bn8ALGHS306Xm+uIYgqMBbjLY/saqs1BCd7LC2Z56YNqXexWE3hSvHctCEBkSlUk6WBQGv9P611T611G611N611qpXtEb4TF+1gd7tObIxNZFTB6urj/ljB6hPbtsH+/QE/PwAnxuvdPY3V0UklBbRp5buPhpprCMpbtcEd1dX2vcO0FCfPXD24zkVmJgU8c/XggOoJmGRoSPhFemoSCljaazDDt22gTcVRwPcrWH0mSCaKa1oXY8x1JJUUUFpe4bO79BNrCLaRH+NEq7Bax+3KDJh1TQIrjNXDgRgEQAKB8BNzBeu3iUNoU1nBUPfG6sfs/gFQJzMQDBpkbTu8JCMzl+LWHShpF80Az4RxeUUlsxesb+CZTWfOPfTdU1S9kKzmcTtLS3GyetZ4np82BGe0A4WxIC8Qh4Nqkuqjwm+c0Q6+jx9ERVg4F2xdzXeeGkSB8AFwiuxso3RyVJTVLfEKMxjndEkkaXdB9fHS8gpcWW6v3ummpybxyNs/0PPALuadfQlg0RaZLZCW4gzYu/+6SI9A+E16ahK6XXuy4pIYVWDUHQq0D4BqQVBaoiYzGOfG9qL/7kLCqk7U4vfF0F3SfmPIKa9zPNGOCFuuIQglEgiE35hjrGuThpO8I48BrSsC8wPg6FHIzQ2KjCGTGYxzYxOJPH6MXqUnKpF6c+jOzBjq6ja2p8zrHM/R43WXaBD+I4FA+FVaipObH/4VYWg+G1wZeEEAICcHjh8Pqh5BWoqTTm0jyIlNBGqUmsC7Q3cnMoa2cVyFURATZ/uMoVAggUD437nnGhvZL1pkdUuazJXl5pE/vw3AjOWHAzP1tR6zJg1kW49EqlDVpSa8PXRn9i767Slka6ceVIRH1DourCGTxcL/IiLg4osDLhCYwxq/37qJo+GtWB7RhR8DvUxGDeY1bHsljqSSrTh9sL90ddXR3YGXMRTMpEcgrDFuHOTlccUfA6cI3ewF6ymvqOTMkgJ+6tKLyrDwoBvWSEtxknDRCCbqEpbe5/1y4empSXQI0/Qq3V4dCAI2YSCISCAQlvjCaUy09luzDA3VRejsGgxcWW5Ky4368gNKCsiN7VX9WNANayQnG3sxl5V5/aXTUpw8N7Q9EVWVtq46GmokEAhLPPSTprhDFy7esqr6mJ3vrs2tCTuV7afbob1s9EyqQhAOayQnG7WUNmzwycuPU3sBeHb2dJ/0OkTTSSAQlijef4QlvYcyqmA1EZUndnKy6921uduUueo2J/aM6seCbljDTIvNzvbN62/0rCofMMA3ry+aTAKBsERctIOveg+lw7HyWuUm7H53baZV5tboEQTdHW2fPuBweD0QuLLcxlzQ3EXsiOqK66f9Xn190XwSCIQl0lOTWNX3HCrCwrk43xgesvOkoVlobEBJAbvbRlHSLrrW8aASHg5nneXVQGBmXLlLy+mzp4hNnZy2nhMKNRIIhCXSUpw8OH0k2YmDuDh/le0nDWdPHkhEmGJAyRZjolgpIsJUQO1C1STJyV4NBOZCMqWr6LPX2KfYznNCoUYCgbBMWoqTc2691li8VLSNO+etps/MT3jQ5aOx6RZIS3GSMXUQSbuLyIk9A2e0g4yrArP2fKMkJ8POnVBS4pWXM+d+4g7spm3FUfK6xNc6LqwlgUBY6u9t+gBwQf5KACq15s3lhfYMBlFHcVQc4abbpwR/touXJ4xrlp4GqtcQ2H1OKFRYvVVlhlIqRym1Vin1P6VUtJXtEf73/PZItrfvXD1PYHprRZFFLaqbK8vN/Q/PBeCmVUeCfmz7U7oA8PDjb3tlsV96ahIR4apWIIgIV7adEwo1VvcIFgKDtNZnA5uAmRa3R/hZJVSnkbaqPH7iuGcDdTuorphZkEulCmNpG3vvsdtSriw3f/hqB3scHUkq2eq9xX4a+uwpYq+jI3vbRhnb0wlbsHrP4s+11uZv/3Kgp5XtEf4XrhRLeg+j47EyzinOqXXcLsyJzgElBRR0iuNIRGRQT3RmZOZSfryK3NjE6uJzLb3ejMxcKqq0Z1cy49e8okoH7d9hoLG6R1DT/wGf1vegUuoWpdRKpdTKEi9NYAnrTR8Rz9LEIZ400pW1jtuFOaGZVFJATjCXlvAwrys3thf9dheidFWt4y15zb57ttUqNhesf4eBxueBQCm1SCm1ro6vKTXOeQA4Dsyp73W01i9prYdprYfFxsb6utnCTx5LSybtogGscp7FxfmrCFeK60Ym2Gr/17hoB22PldNr347qev3m8WBkXldObCLtKo4QX7oTgOi2zV8zEeWIIKZsPzHlB9gsVUdtx+eBQGs9Tms9qI6vDwCUUjcClwMztLbRwLDwm8fSkhn5mxmctWsLm39rv03ARw+IJWl3IWHo6hXFdl781lLmxK55rebw0KEjx5s1T+DKcnP42PFTMoYiwmSy2C6szhqaANwLTNZae7/UoQgcEycaf372mbXtOIkry837q9z093wYbux6Bgq4YmhwbV5eU1qKk3atW7GpSwJwoqxGc8f0MzJzqajUpwSC9pGtgvbvMNBYPUfwD6ADsFAptVop9aLF7RFWSU4GpxM+rXeayBI1J4oPtXawLaorGlicE9zzVPvLKyhr7WBrdHeSPIX2wCgX3lTV8wO7iyiLaENxRyM1tbSs4nRPE35kddZQX611vNZ6iOfr11a2R1hIKZgwARYuNPYDtgnzQ2zw9k3kxCaiVVit48HKHLvPjU3krF351ccVNHl4qOZiss0xPav/DmV+wD6s7hEIccLEibB/PyxbZnVLqsVFO4guP8CQ4k18mzik1vFglp6ahAK+SRxC733FDNqRBxip/00dHho9IBYF9KmRMRTMcyyBSAKBsI9x46BVK1sND6WnJjG2cA1haJb0HgaExodYWooTDXxw1sWUt2rD9DUn5m6a0hsy51jO3JmP82AJWXFJQT/HEogkEAj7iIqC88+3VSBIS3Fy57GfKG3bkezufW1fJdWbnNEODkS256MBP2PKhq9od9TI52hKb8icY7l29accadUa18DRITHHEmgkEAh7mTgRVq+G7dutbomhqor4H74h+ueT2PzU5OAvNldDemoSjohw5g6ZQPtj5Uza+HWTe0PFpeW0PVbOlA1L+HjABRyIbF99XNiHBAJhLxMmGH/aJY101SqjFLOZ3hpC0lKcPDE1mV1nDSEnNpEb133e5N5QXLSDyRu+osOxcuYMnljruLAPCQTCXgYPhl69YN48q1ti+PRTI6MpNdXqllgiLcVJ+oQBfHzeJAa4N7HgtQ+blDWUnprEdWszyenSix+dxh7FoTDHEmgkEAh7UQp+8QsjjbS42LJmmPvrZr04l/XOJFxFRy1ri5XMyqv/OWMU5a3aMPbbD5pUiTStageDtv/EJ+dNRikVUnMsgUQCgbCf66+HqirW/eUFRj35JWfc97FXauI3lvnhV1a8g8HbN/F5r3OCuuz06ZiTvTUnjcMOHWx8Cum//gUOB3944xG2PHlZSM2xBBIJBMJ++vVjz+BhRM59E/e+MjR4ryZ+I5gffj8ryPKkjQ4N6rLTp1NzUrfmpHGjJnsPHIC5c+GaayA62neNFC0mgUDY0iu9f0bf3YUkexYyQctr4jeW+SF3cf4q9jg6srZHv1rHQ0nNSd2suCQ2xiZy7ZrPGjfZO3cuHD4Mt97qwxYKb5BAIGxpTsIIjoZHcMW6L2od98eHcVy0A6WruCh/FV+fcU5Il0QwU0gBUIq5QyZw9o48HotvYM5Ea2NYaPBgOPdc3zdUtIgEAmFL4TGd+LzfSKZs+IrWx08UJ2tJTfzGGj0glsHbf6Jz+QEW9wmd1cR1MVNIndEOFPD9eRM5HhnJ6K9dp3/iDz8Y60FuvdVIABC21srqBghRF63h/UFjmZTzDaM3/0Bm0vnVx33JLIlwS/4qqlB8k5gS8iUR0lJOuvbCa4xhn6efhg4d6n7Sv/4F7drBjBn+aaRoEekRCFvaX17BN2eksLN9DFfWGB4qLfdt6WJzovji/FWs6dGffW2jpCTCyW69FQ4dgrffrvvx/fuNx6ZPh44d/ds20SwSCIQtxUU7qAwL539nXczF+SuJKdsPNK8MclMUl5YTU7afwds3saT30FrHhWd9xeLD5MQmsmnWU3z+4XdQVVXrnDVP/APKyph8bKBf035F80kgELZklkF+f9BYIqoqmbJhCdC8MshNERft4GdbfiQMXT0/YB4Pdeb6Cvf+I7w6bDL9t+cxfvIoKjpGwahRcNttrL7/Cdq9+hLZ3fqwtkc/v6b9iuazeqvKR5VSaz27k32ulIqzsj3CPswyyD/F9mJt975cse7L6sd8eXeenprE2IIf2d02iuzufYHQnSg+mTlsBvDu2eO5/IbnuS/1Dj44exyEh8NbbzHkifvpu7uQOUNO1BUK1TUYgcTqyeIMrfWfAJRSvwMeAmSXMgEYZZDdpeW8P2gsDy/6F0klBeTGJvr07jzt7O4c3baGxf2HgwrDGe0gPTUpZCeKazo5AK/r3pd13fsyD7jyyctw/biNp15ZSHzpTr6PH3ja5wp7sXqrygM1fmyH0fMXAjiRw77gzAs5FtaKK7K/8P3d+cqVtCndy4R7/09KIpykvgBsHs/4fBPFHbuyIiG5eu1FQ88V9mD5HIFS6s9KqSJgBkaPoL7zblFKrVRKrSwpkQyOUGDmsLeN687iPsOYunEJbZXmrnmrfTcJ+emnEBYG48d7/7UDXK3FZR4KY90FnH5jexlaszefBwKl1CKl1Lo6vqYAaK0f0FrHA3OAO+p7Ha31S1rrYVrrYbGxsb5utrCJtBQnS+8bQ/Svb6bLoX0kb1zh29pDn3xirITt3Nm7rxsE0lKcXDHUSc3lYRp4f5UbV5ab8HoWjimF9KpszueBQGs9Tms9qI6vD046dQ5wha/bIwLTvWU92ePoyIzVJ7ax9Pok5JdfGitir77ae68ZZBbnlJwyflteUcmd81ZTWc9qP18vAhQtZ3XWUL8aP04Bcqxqi7C3rYeO8+rwNC7J+56L8ldVHz/dcESTVFbC3Xcbm+Lcdpt3XjMINWfS1ynzA7Zn9RzBk55horXAeOD3FrdH2FS4Urwy/OdsjunJIwv/SZuKE0XPvDI89N//GrVxnngCIiNb/npBqqmTvpJ6Gxiszhq6wjNMdLbWepLWWladiDpVas2xVhH86ZJf06t0B7cvf6/6sZnz17bsxQ8fhgceMOYGrrmmhS0NbnVNGNdHdiMLHFb3CIRoFHN44bvEIXxw5kX8esW7JO417hvKK6p40JXd/Bd/5hljW8xnn5VKmQ2oWY30dJzRDkm9DSASCERAMEtOADw25iaOhrfmkYUvVs9EvrWiqHkvvH07R594kk+TRpH4YSl9Zn7SsqASAsxMruenDSEi7NTAGRGuZDgowEggEAEhLcXJjJEJAJS0j+GZC3/BhQVZXJbzLUC9GSsNWXn9b1DHKnjiohurX+fN5YUSDBohLcVJxlWDiXac2COiU9sIMq4cLD2BAKN0AOZ2DRs2TK9cudLqZggL9J75MVUawqoqWfDGH4g9vI+xN7/IoTZteX7akKZ9AK1dS9XgIbw2bDKPjf1VrYfClWLzE5d6ufVCWEsptUprPezk49IjEAHl2hFGr6AqLJwHxt9O7KF93PXtHAAe/nB9419Ia/Ku/zUHItvx9/NPnSBubg9DiEAkgUAElMfSkqu/XxOXxNwhE7hx1YectTOffWUVjU8l/ewz+q5Zxt/On85+x6m7bNW3SlaIYCSBQAScmhkrT110A/scHXjm42fpX1LA7AWN6BXs2gX33MOWTj347zl1D/9MHxHvreYKYXsSCETAqZmRciCyPemX3kncgRI+ff13pLue49NFq+t+4r59xnqB3r0hJ4fHxt5CRXhEnafW7HkIEewkEIiAk5birJWpsrjPcC669WXeOOcyrl67kJ9dPgqefBKOHDFOOHQIHn/cCACPPw6XXw4bNtBjRt2lra7zZCcJESoka0gEJFeWmzvnrT7leO8925i55DUuyfveqBs0fTq89poxHDRpEjz6KAweXH3+g65s3lpRRKXWhCvF9BHx0hsQQau+rCEJBCJgJd73cb2PFVwSaRSRW7MGxoyBP/8ZRo70Y+uEsJ/6AoHVW1UK0WzRjghKyyvqPM7YsbBqlVE6It6Y+HVlucnIzKW4tJw42YJSiGoyRyAC1uzJA08pcRARppg92bNfbnh4rSAwc3427tJy325sI0QAkh6BCFjm3fzp7vLNXkBd+xaYG9tIr0CEOgkEIqClpTjr/SA3ewHlFZX1Pr85G60IEWxkaEgErYzM3NMGAWj6RitCBCMJBCJoNXS3L7tnCWGwRSBQSt2tlNJKqS5Wt0UEj9Pd7cvuWUKcYHkgUErFY+xXXGh1W0RwqWtbRUdEOM9PGyK7ZwlRg+WBAHgOuBcIvJVtwtZqbquokF6AEPWxNGtIKTUFcGut16gGyv4qpW4BbgFISJBaMKJxTpdVJIQw+DwQKKUWAd3reOgB4H6MYaEGaa1fAl4Co8SE1xoohBAhzueBQGs9rq7jSqlk4AzA7A30BH5USp2rtd7h63YJIYQwWDY0pLXOBrqaPyulCoBhWuvdVrVJCCFCkR0mi4UQQljINiUmtNaJVrdBCCFCUUDuR6CUKgG2NvPpXYBQG36Saw4Ncs2hoSXX3EtrHXvywYAMBC2hlFpZ18YMwUyuOTTINYcGX1yzzBEIIUSIk0AghBAhLhQDwUtWN8ACcs2hQa45NHj9mkNujkAIIURtodgjEEIIUYMEAiGECHFBGwiUUhOUUrlKqTyl1H11PN5GKTXP8/gKpVSiBc30qkZc8x+UUhuUUmuVUl8opXpZ0U5vauiaa5x3hWfzo4BONWzM9Sqlrvb8O69XSs31dxu9rRH/rxOUUouVUlme/9uXWtFOb1JKvaaU2qWUWlfP40op9TfP38lapdQ5LXpDrXXQfQHhwGagN9AaWAOcddI5twMver6/Bphndbv9cM2jgbae728LhWv2nNcB+BpYjlHPyvK2+/DfuB+QBXTy/NzV6nb74ZpfAm7zfH8WUGB1u71w3RcC5wDr6nn8UuBTQAEjgRUteb9g7RGcC+RprfO11seAt4EpJ50zBfiP5/v3gLGqoU0R7K3Ba9ZaL9Zal3l+XI5R8TWQNebfGeBR4C/AEX82zgcac72/Al7QWu8D0Frv8nMbva0x16yBjp7vo4BiP7bPJ7TWXwN7T3PKFOANbVgORCulejT3/YI1EDiBoho/b/Mcq/McrfVxYD/Q2S+t843GXHNNN2HcUQSyBq/Z02WO11p/7M+G+Uhj/o37A/2VUkuVUsuVUhP81jrfaMw1zwauU0ptAz4Bfuufplmqqb/vp2WbonPCf5RS1wHDgIusbosvKaXCgGeBGy1uij+1whgeuhijx/e1UipZa11qZaN8bDrwb631M0qp84D/KqUGaa2rrG5YoAjWHoEbiK/xc0/PsTrPUUq1wuhS7vFL63yjMdeMUmocxu5wk7XWR/3UNl9p6Jo7AIOAJZ79LkYCCwJ4wrgx/8bbgAVa6wqt9RZgE0ZgCFSNueabgHcAtNbLgEiMwmzBrFG/740VrIHgB6CfUuoMpVRrjMngBSedswC4wfP9lcCX2jMLE6AavGalVArwL4wgEOhjx9DANWut92utu2itE7VR5nw5xrWvtKa5LdaY/9cujN4ASqkuGENF+X5so7c15poLgbEASqkzMQJBiV9b6X8LgOs92UMjgf1a6+3NfbGgHBrSWh9XSt0BZGJkHbymtV6vlHoEWKm1XgC8itGFzMOYlLnGuha3XCOvOQNoD7zrmRcv1FpPtqzRLdTIaw4ajbzeTGC8UmoDUAmka60DtqfbyGu+G3hZKXUXxsTxjQF+U4dS6i2MgN7FM/cxC4gA0Fq/iDEXcimQB5QBv2zR+wX435cQQogWCtahISGEEI0kgUAIIUKcBAIhhAhxEgiEECLESSAQQogQJ4FACCFCnAQCIYQIcRIIhPACTz38SzzfP6aU+rvVbRKisYJyZbEQFpgFPKKU6gqkAAG7YluEHllZLISXKKW+wijhcbHW+qDV7RGisWRoSAgvUEolAz2AYxIERKCRQCBEC3l2hpqDsWvUoSDYDEaEGAkEQrSAUqotMB+4W2u9EWNbzFnWtkqIppE5AiGECHHSIxBCiBAngUAIIUKcBAIhhAhxEgiEECLESSAQQogQJ4FACCFCnAQCIYQIcf8fJdH0yINBEZsAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light", "tags": [] }, "output_type": "display_data" } ], "source": [ "X_grid = torch.from_numpy(np.linspace(0,1,50)).float().view(-1, d)\n", "y_hat = neural_network(X_grid)\n", "plt.scatter(X.numpy(), y.numpy())\n", "plt.plot(X_grid.detach().numpy(), y_hat.detach().numpy(), 'r')\n", "plt.title('plot of $f(x)$ and $\\hat{f}(x)$')\n", "plt.xlabel('$x$')\n", "plt.ylabel('$y$')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "id": "mo4LEXsyJhxU" }, "source": [ "## Useful links:\n", "- [60 minute PyTorch Tutorial](https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html)\n", "- [PyTorch Docs](https://pytorch.org/docs/stable/index.html)\n", "- [Lecture notes on Auto-Diff](https://courses.cs.washington.edu/courses/cse446/19wi/notes/auto-diff.pdf)\n", "\n" ] } ], "metadata": { "colab": { "name": "PyTorch Introduction.ipynb", "provenance": [] }, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.10" } }, "nbformat": 4, "nbformat_minor": 1 }