{ "cells": [ { "cell_type": "markdown", "id": "subject-defense", "metadata": {}, "source": [ "# PyTorch Introduction - Neural Networks\n", "\n", "In this notebook we build on top of the previous notebook, and show how to build, train and evaluate neural networks in PyTorch." ] }, { "cell_type": "markdown", "id": "latin-payroll", "metadata": {}, "source": [ "## Import and Helper Functions\n", "\n", "Here we import torch, define helper function for visualization and define dataset for later use." ] }, { "cell_type": "code", "execution_count": 1, "id": "foster-sterling", "metadata": { "id": "0WbwvskP4spn" }, "outputs": [], "source": [ "import torch\n", "import torch.nn as nn\n", "import torch.nn.functional as F\n", "\n", "from mpl_toolkits.mplot3d import Axes3D\n", "import matplotlib.pyplot as plt\n", "\n", "import numpy as np\n", "\n", "torch.manual_seed(446)\n", "np.random.seed(446)" ] }, { "cell_type": "code", "execution_count": 2, "id": "developmental-exemption", "metadata": { "id": "9jISIWlg4sqF" }, "outputs": [], "source": [ "def visualize_fun(w, title, num_pts=20):\n", " \n", " x1, x2 = np.meshgrid(np.linspace(-2,2, num_pts), np.linspace(-2,2, num_pts))\n", " X_plane = torch.tensor(np.stack([np.reshape(x1, (num_pts**2)), np.reshape(x2, (num_pts**2))], axis=1)).float()\n", " y_plane = np.reshape((X_plane @ w).detach().numpy(), (num_pts, num_pts))\n", " \n", " plt3d = plt.figure().gca(projection='3d')\n", " plt3d.plot_surface(x1, x2, y_plane, alpha=0.2)\n", "\n", " ax = plt.gca()\n", " ax.scatter(X[:,0].numpy(), X[:,1].numpy(), y.numpy(), c='r', marker='o')\n", "\n", " ax.set_xlabel('$X_1$')\n", " ax.set_ylabel('$X_2$')\n", " ax.set_zlabel('$Y$')\n", " \n", " plt.title(title)\n", " plt.show()" ] }, { "cell_type": "code", "execution_count": 3, "id": "operating-prospect", "metadata": { "id": "SVEHCLMd4sqC" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "X shape torch.Size([50, 2])\n", "y shape torch.Size([50, 1])\n", "w shape torch.Size([2, 1])\n" ] } ], "source": [ "# make a simple linear dataset with some noise\n", "\n", "d = 2\n", "n = 50\n", "X = torch.randn(n,d)\n", "true_w = torch.tensor([[-1.0], [2.0]])\n", "y = X @ true_w + torch.randn(n,1) * 0.1\n", "print('X shape', X.shape)\n", "print('y shape', y.shape)\n", "print('w shape', true_w.shape)\n" ] }, { "cell_type": "markdown", "id": "opposed-operations", "metadata": { "id": "9Jx1VRet4sqL" }, "source": [ "## torch.nn.Module\n", "\n", "`Module` is PyTorch's way of performing operations on tensors. Modules are implemented as subclasses of the `torch.nn.Module` class. All modules are callable and can be composed together to create complex functions.\n", "\n", "[`torch.nn` docs](https://pytorch.org/docs/stable/nn.html)\n", "\n", "Note: most of the functionality implemented for modules can be accessed in a functional form via `torch.nn.functional`, but these require you to create and manage the weight tensors yourself.\n", "\n", "[`torch.nn.functional` docs](https://pytorch.org/docs/stable/nn.html#torch-nn-functional).\n", "\n", "### Linear Module\n", "The bread and butter of modules is the Linear module which does a linear transformation with a bias. It takes the input and output dimensions as parameters, and creates the weights in the object.\n", "\n", "Unlike how we initialized our $w$ manually, the Linear module automatically initializes the weights randomly. For minimizing non convex loss functions (e.g. training neural networks), initialization is important and can affect results. If training isn't working as well as expected, one thing to try is manually initializing the weights to something different from the default. PyTorch implements some common initializations in `torch.nn.init`.\n", "\n", "[`torch.nn.init` docs](https://pytorch.org/docs/stable/nn.html#torch-nn-init)" ] }, { "cell_type": "code", "execution_count": 4, "id": "coordinated-special", "metadata": { "id": "GD_LYlyI4sqL" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "example_tensor torch.Size([2, 3])\n", "transormed torch.Size([2, 4])\n", "\n", "We can see that the weights exist in the background\n", "\n", "W: Parameter containing:\n", "tensor([[ 0.3270, 0.2183, 0.2269],\n", " [-0.5094, -0.4306, 0.2483],\n", " [-0.0776, -0.5372, 0.0966],\n", " [-0.1610, 0.2270, -0.0063]], requires_grad=True)\n", "b: Parameter containing:\n", "tensor([ 0.1384, -0.1959, -0.2587, 0.0353], requires_grad=True)\n" ] } ], "source": [ "d_in = 3\n", "d_out = 4\n", "linear_module = nn.Linear(d_in, d_out)\n", "\n", "example_tensor = torch.tensor([[1.,2,3], [4,5,6]])\n", "# applys a linear transformation to the data\n", "transformed = linear_module(example_tensor)\n", "print('example_tensor', example_tensor.shape)\n", "print('transormed', transformed.shape)\n", "print()\n", "print('We can see that the weights exist in the background\\n')\n", "print('W:', linear_module.weight)\n", "print('b:', linear_module.bias)" ] }, { "cell_type": "markdown", "id": "raised-bleeding", "metadata": { "id": "VLy9acBr4sqN" }, "source": [ "### Activation functions\n", "PyTorch implements a number of activation functions including but not limited to `ReLU`, `Tanh`, and `Sigmoid`. Since they are modules, they need to be instantiated." ] }, { "cell_type": "code", "execution_count": 5, "id": "ancient-drinking", "metadata": { "id": "ZkEmm9wj4sqN" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "example_tensor tensor([-1., 1., 0.])\n", "activated tensor([0., 1., 0.])\n" ] } ], "source": [ "activation_fn = nn.ReLU() # we instantiate an instance of the ReLU module\n", "example_tensor = torch.tensor([-1.0, 1.0, 0.0])\n", "activated = activation_fn(example_tensor)\n", "print('example_tensor', example_tensor)\n", "print('activated', activated)" ] }, { "cell_type": "markdown", "id": "extraordinary-rebate", "metadata": { "id": "yZLWExSi4sqO" }, "source": [ "### Sequential\n", "\n", "Many times, we want to compose Modules together. `torch.nn.Sequential` provides a good interface for composing simple modules." ] }, { "cell_type": "code", "execution_count": 6, "id": "initial-outdoors", "metadata": { "id": "AiEuwl7q4sqO" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "transformed torch.Size([2, 1])\n" ] } ], "source": [ "d_in = 3\n", "d_hidden = 4\n", "d_out = 1\n", "model = torch.nn.Sequential(\n", " nn.Linear(d_in, d_hidden),\n", " nn.Tanh(),\n", " nn.Linear(d_hidden, d_out),\n", " nn.Sigmoid()\n", " )\n", "\n", "example_tensor = torch.tensor([[1.,2,3],[4,5,6]])\n", "transformed = model(example_tensor)\n", "print('transformed', transformed.shape)" ] }, { "cell_type": "markdown", "id": "respected-mobility", "metadata": { "id": "vX22VrSU4sqQ" }, "source": [ "Note: we can access *all* of the parameters (of any `nn.Module`) with the `parameters()` method. " ] }, { "cell_type": "code", "execution_count": 7, "id": "caroline-cartoon", "metadata": { "id": "nE40cYPA4sqQ" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Parameter containing:\n", "tensor([[ 0.5478, -0.5734, 0.2589],\n", " [ 0.5739, -0.4392, -0.0377],\n", " [ 0.2290, 0.0529, 0.4021],\n", " [ 0.3153, -0.4802, 0.3067]], requires_grad=True)\n", "Parameter containing:\n", "tensor([ 0.4905, 0.3743, 0.4069, -0.2514], requires_grad=True)\n", "Parameter containing:\n", "tensor([[-0.1443, -0.1406, 0.0414, -0.4699]], requires_grad=True)\n", "Parameter containing:\n", "tensor([0.2149], requires_grad=True)\n" ] } ], "source": [ "params = model.parameters()\n", "\n", "for param in params:\n", " print(param)" ] }, { "cell_type": "markdown", "id": "cordless-ladder", "metadata": { "id": "B-5ELoCR4sqS" }, "source": [ "### Loss functions\n", "PyTorch implements many common loss functions including `MSELoss` and `CrossEntropyLoss`." ] }, { "cell_type": "code", "execution_count": 8, "id": "second-referral", "metadata": { "id": "T5mVlwlZ4sqS" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tensor(0.6667)\n" ] } ], "source": [ "mse_loss_fn = nn.MSELoss()\n", "\n", "input = torch.tensor([[0., 0, 0]])\n", "target = torch.tensor([[1., 0, -1]])\n", "\n", "loss = mse_loss_fn(input, target)\n", "\n", "print(loss)" ] }, { "cell_type": "markdown", "id": "accomplished-inquiry", "metadata": { "id": "O9_XAc4s4sqT" }, "source": [ "## torch.optim\n", "PyTorch implements a number of gradient-based optimization methods in `torch.optim`, including Gradient Descent. At the minimum, it takes in the model parameters and a learning rate.\n", "\n", "Optimizers do not compute the gradients for you, so you must call `backward()` yourself. You also must call the `optim.zero_grad()` function before calling `backward()` since by default PyTorch does and inplace add to the `.grad` member variable rather than overwriting it.\n", "\n", "This does both the `detach_()` and `zero_()` calls on all tensor's `grad` variables.\n", "\n", "[`torch.optim` docs](https://pytorch.org/docs/stable/optim.html)" ] }, { "cell_type": "code", "execution_count": 9, "id": "empty-laugh", "metadata": { "id": "uG5Roarn4sqT" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "model params before: Parameter containing:\n", "tensor([[-0.3881]], requires_grad=True)\n", "model params after: Parameter containing:\n", "tensor([[-0.3603]], requires_grad=True)\n" ] } ], "source": [ "# create a simple model\n", "model = nn.Linear(1, 1)\n", "\n", "# create a simple dataset\n", "X_simple = torch.tensor([[1.]])\n", "y_simple = torch.tensor([[2.]])\n", "\n", "# create our optimizer\n", "optim = torch.optim.SGD(model.parameters(), lr=1e-2)\n", "mse_loss_fn = nn.MSELoss()\n", "\n", "y_hat = model(X_simple)\n", "print('model params before:', model.weight)\n", "loss = mse_loss_fn(y_hat, y_simple)\n", "optim.zero_grad()\n", "loss.backward()\n", "optim.step()\n", "print('model params after:', model.weight)\n" ] }, { "cell_type": "markdown", "id": "wooden-phrase", "metadata": { "id": "jpREGIlv4sqV" }, "source": [ "As we can see, the parameter was updated in the correct direction" ] }, { "cell_type": "markdown", "id": "informal-compression", "metadata": { "id": "IZrHzKoh4sqV" }, "source": [ "## Linear regression using GD with automatically computed derivatives and PyTorch's Modules\n", "\n", "Now let's combine what we've learned to solve linear regression in a \"PyTorchic\" way." ] }, { "cell_type": "code", "execution_count": 10, "id": "incoming-committee", "metadata": { "id": "qBgIN7VW4sqV" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "iter,\tloss,\tw\n", "0,\t3.45,\t[-0.6277163 0.5246437]\n", "1,\t2.23,\t[-0.6903032 0.81286395]\n", "2,\t1.45,\t[-0.74182737 1.0444099 ]\n", "3,\t0.94,\t[-0.7842876 1.2304163]\n", "4,\t0.61,\t[-0.8193146 1.3798317]\n", "5,\t0.40,\t[-0.8482402 1.4998472]\n", "6,\t0.26,\t[-0.8721527 1.5962417]\n", "7,\t0.17,\t[-0.8919422 1.6736592]\n", "8,\t0.11,\t[-0.9083374 1.7358311]\n", "9,\t0.08,\t[-0.92193544 1.785756 ]\n", "10,\t0.05,\t[-0.93322587 1.825843 ]\n", "11,\t0.04,\t[-0.9426106 1.8580279]\n", "12,\t0.03,\t[-0.95041996 1.8838661 ]\n", "13,\t0.02,\t[-0.95692545 1.9046069 ]\n", "14,\t0.02,\t[-0.9623507 1.9212543]\n", "15,\t0.01,\t[-0.9668801 1.9346145]\n", "16,\t0.01,\t[-0.9706655 1.9453354]\n", "17,\t0.01,\t[-0.97383255 1.953937 ]\n", "18,\t0.01,\t[-0.976485 1.9608375]\n", "19,\t0.01,\t[-0.9787088 1.9663724]\n", "\n", "true w\t\t [-1. 2.]\n", "estimated w\t [-0.9787088 1.9663724]\n" ] } ], "source": [ "step_size = 0.1\n", "\n", "linear_module = nn.Linear(d, 1, bias=False)\n", "\n", "loss_func = nn.MSELoss()\n", "\n", "optim = torch.optim.SGD(linear_module.parameters(), lr=step_size)\n", "\n", "print('iter,\\tloss,\\tw')\n", "\n", "for i in range(20):\n", " y_hat = linear_module(X)\n", " loss = loss_func(y_hat, y)\n", " optim.zero_grad()\n", " loss.backward()\n", " optim.step()\n", " \n", " print('{},\\t{:.2f},\\t{}'.format(i, loss.item(), linear_module.weight.view(2).detach().numpy()))\n", "\n", "print('\\ntrue w\\t\\t', true_w.view(2).numpy())\n", "print('estimated w\\t', linear_module.weight.view(2).detach().numpy())" ] }, { "cell_type": "code", "execution_count": 11, "id": "north-shock", "metadata": { "id": "RKZPbcUw4sqW" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQsAAAEDCAYAAAAiBiE/AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAACX/klEQVR4nOy9d5wkZ3Xu/32r83RPh4k7aXMO0kqrFRIZDAgDQsIkEYwu2L74GmxjLljX/mF/4PraYBww5tok29hwbTBIwgKRwQYMMsraHCbs5DydpnN31fv7o8JU93TP9MzO7K5EP/ai6e7qqreq6z113nOe8xwhpaSBBhpoYDUoV3sADTTQwNMDDWPRQAMN1IWGsWiggQbqQsNYNNBAA3WhYSwaaKCButAwFg000EBdaBiLBhpooC40jEUDDTRQFxrGogqEEGeEEC9c4fNhIcRL1rC/NW1/tSCE+EchxP/ZhP1+WAjxno3e72bhav1eQohHhBCHrvRx60VdxsK4eFkhxKIQIi6EeEgI8etCiLV8f9Mv/kYdR0p5SEr5w43c588rhBDtwNuATxuvzXspJYSYEUJ8TggRqGM/Kds/zbaPlBDiLZt9HquM7S4hxMNCiLQQYtb4+zeEEMK2TT1z6M+B/33lz6A+rMWzuF1K2QxsAz4C3AP8/aaMqoFVIYRwXu0x1In/BnxTSpm1vXe7lDIA3AgcBz6w2k6klAHzHzBq7sP498/1Dmajr5sQ4n8CHwf+DNgCdAK/DjwHcFdsvtoc+hrwIiFE10aOccMgpVz1HzAMvKTivZsBDThsvP5fwCCwCJwFXmO8/wVjuyyQAn53pe2Nz+4BJozPLgC/YLzfDdwHzAGXgN+yfafqcWyfvx34uu31APBl2+sx4Kj9fFcY+zDwPuAkkAD+FfDWc/1WOYea18S2n3uM4+YB50pjWelYxuc3AE8Yx/tX4EvA/6lxDimgz/j71wEJdBqv3w/8XY3v/Tvw1lr3Evoke9C2n/sqvv8J4K/quB8PAD8E4sAZ4NWrXLc+4H7j2iwA/7di+1V/XyAEpIHXbsQcMt77HnB3PfPySv9bt7Ew3h8F/ofx9+uNm1MB3mhcxK4VLlTV7YF96BO329huO7DL2O5x4A/RLfZOYAi4bbVxGp/tNG4kxTjOCDBh+ywGKFUmd7WxDwOPGONvAc4Bv77a9VvtHFa6hrb9PGXc6L6VxlLHsdzGNfgdwAW8DihS21iMAQcBAZwC+oH9xusB4Poa35sDjlf7jYzzOAP8kfG6yzjnsPHaCcwCx1a6H43xDwC/b5zXi9EN4L5q1w1wACeAjwF+wAs8d62/L/ByoAQ4N2IOGa//GvjLq20Yqv273ADnpHExkVJ+RUo5KaXUpJT/in4z3VzriytsrwIe4KAQwiWlHJZSDqK7q+1Syv8tpSxIKYeAzwJ31TNQY/tF4CjwAuA7wIQQYr/x+j+llNoazv2vjfFHga8b+10NK55Dndfwr6WUY7Lcra82ltWu1y3ok+yvpJRFKeW9wKMrjD0OBICXoU/M00AYfcJMSClP1PheGP262/FvQog48BPgR8CfGOc/BfwY3Whi7HteSvn4CuMyzyUAfMQ4138HHgTeZNvGft1uRjcE75dSpqWUOSnlTyr2Wc/v22aMr2S+YcQi4kZ84vmrjBtsc8jAIvo1u+Zwueu3HiAKIIR4G/BedE8A9B+vrdYXa20vpRwwIucfBA4JIb5jbLcN6DZuMhMO4D/XMN4fAS8Edht/x9ENxa3G67Vg2vZ3Bv3mWw0rnkOd13CszrGsdr260Se5XaNgZIWxx4zxvAf4U/RlXQT4DfSn4Urfa654704p5fdrbP9PwP9AN2xvRV8KroZuYKzC2I+g358m7NetDxixT/IqqOf3XQDahBBOc19SymcDCCHGqS8maM0hA83o9+U1h3V7FkKI4+gn+hMhxDb0H/fdQKuUMoz+5DGjwbLiuytuL6X8Fynlc9FveIl+c44Bl6SUYdu/ZinlK2y7Xk2cwzQWzzP+/hG6sXgBtY3FRgp+1DyHOq7hWsez2vWaAnrsEXtg6wr7iwM3oS+LfggkgeuBw8C/rfC9k8DeOseMsa/rhBCHgVcB9QQvJ4G+iszCVvS4lwn7dRsDtm5AsPO/0GMgd6zny/Y5ZHv7APoS6ZrDmo2FECIohHgVejDs/0kpT6Gv+yT6+hQhxNvRbyITM+hrZhM1txdC7BNCvFgI4QFy6MFFFX0NmRRC3COE8AkhHEKIw8YFr3WcSvwIeBH6en8c/Sn7cqAVeLLGd1bb51qw0jmsdg038lig3+gl4LeEEE4hxC+xwrIR3UP4HZa8iCTw28CnpZTqCt/7JroxrgtSyhxwL/AvwCNSytE6vvYweqzjd4UQLoMjczv6PVoNj6Aby48IIfxCCK8Q4jn1jtE21jjwIeBvhRCvE0IEhBCKEOIo+u9ZFTXmEMY9fww9yHnNYS3G4utCiEV0q/z/AX+J7ooipTwL/AX6DTgDHAF+avvuh4EPGGu5962yvQc9rTSP7gp2AL9v3JC3o68dLxmf/x16RLrqcSpPQEp5ET2q/5/G6yR60O+nK9zwK+5zLVjpHOq4hht2LOPzAvBL6KnNGHpA9f4VdhlDX7b+i/E6ib62/uwqQ/k88AohhG8Nw/8n9POvZwlinsurgV9EP8+/Bd4mpTxfY3vz2uxGDzCOo5//miGl/Cj60vF30YOxM+icknuAhyo2rzmHDLwa+KGUcnI9Y9lsiPIlawMNbDyEEH8CzEop/6rO7bcC54EthkH/uYAQ4mHgV6SUp6/2WKqhYSwauKZgxB3+EghKKd9xtcfTwBKeLizABn4OIITwo7vxI+ixpAauITQ8iwYaaKAuNKpOG2iggbrQMBYNNNBAXVhLzKKxXmmggc1HJQnvmkHDs2iggQbqQsNYNNBAA3WhYSwaaKCButAwFg000EBdaBiLBhpooC40jEUDDTRQFxrGooEGGqgLDWPRQAMN1IWGsWiggQbqQsNYNNBAA3WhYSwaaKCButAwFg000EBdaBiLBhpooC40jEUDDTRQFxrG4hpAQ62sgacDGhqcVxmappHJZFBVFbfbjdPpxOFwoCgNO97AtYW1aHA2Hn8bCCklpVLJ+qeq5W1LFEXB5XI1jMfPH65Z8ZuGsbgK0DSNYrGIpmkIISiVStbfUN7ZPhaLoWkaHR0duFwuHA4HTqfT2raBZxyu2R+2sQy5gpBSoqoqxWIRACFE1Ulvf79YLFqGJZfLWds4HI4yz6NhPBrYbDSMxRWCfdlRy0hUg7mdfRkipWwYjwauOBrG4gqgctlxuRO5ch8N49HAlUDDWGwi7MsOIcS6gpRCiFVTq7WMRzabtd5vGI8GLhcNY7FJkFJSKBTq8iYURUHTtA07tnk80zhVMx6KouBwOPB4PA3j0UBdaBiLTYCmaQwPD7NlyxYURVlxIpZKJS5cuICUkpaWFsLhME7n0s9Sj2exGqoZj0QiweTkJPv27QPA6XRansdqY27g5xMNY7GBsAcxR0dH6erqWnHSLS4ucurUKXp6enC73cTjcUZGRgAIh8NEIpEN9ThMmIbD9C7M5VKpVEJKab3fMB4N2NEwFhuEymWHubSoFqeQUjI2NsbExATXXXcdXq8XVVVpbW0FdG8jHo+zsLDA/Py8xbkIh8OEQqENJ2hVi3mYxsP83Ol0Wv8axuPnEw1jsQEolUrLuBO1lg/FYpEzZ87gcrm4+eabcTgc1hPdnIBOp5O2tjba2toIh8MkEgn8fj+zs7MMDAzgdDqJRCK0tLQQCASuiPGoPMeG8fj5Q8NYXAZW4k5UMxaJRILTp0+zc+dOurq66jqG6aV0dHTQ0dEBQD6fJxaLMTExweLiIh6Ph0gkQiQSIRAI1DVx7capnjFUGg+TLGZ+bsY8TGp6w3g889AwFuvEatwJRVEsYyGlZGRkhOnpaW644QaamprqPk41o+PxeNiyZQtbtmwBIJvNEovFGB0dJZVK0dTUZBmPpqamDZ+4QggcDof1utJ4LC4uEgwG8fl8DePxDELDWKwRlZTtWksAc5IXCgVOnTpFU1MTN99886YUhPl8Pnw+H93d3UgpyWQyxGIxhoaGyGQyBAIB3XgEAvg0DbzeDT1+pfGYmpoCsIKz9qI4s66lYTyefmgYizXAfIKqqrrqDS+EsCbsnj17rCXEWrHW1KkQAr/fj9/vp7e3FyklqVSK5NAQ8c9/noVMBo/XS/45z0Ftb1/XmFaDlNIyDObrQqFAPp+3rpvL5bKWLQ3j8fRAw1jUCU3TmJqaoqWlZdWbW0pJOp1meHiYG2+8EZ/PdwVHWg4hBM1+P5Fz55A7dyL9fnLJJIs//Snjx4/zSCpFKBQiEokQDodxu92XfczKeIjd8zANX6FQoFAoALrnURnzaODaQ8NYrAJ7EPPixYs8+9nPXnH7fD7PyZMnATh8+PBlG4qNIGVRKCDSaWR3NwLwhcOIYJD2piZ23HQTyWSSWCzG+Pg4qqpaHI9Kgli9WGm8dvq5fduG8bj20TAWK6CSO7Ea5ufnuXDhAvv27WNycrLu42y6C+7xIP1+SCYhGARjUsqmJhRFIRwOEw6H2bFjB6qqkkgkiMVijIyMWPwO03jYYxMrYa1VtasZj4YQ0NVHw1jUgBnENF3qlW5+TdMYGBggkUhw00034fF4mJ6e3hD25YZ4FkKgvvSlOL7zHZicRCgKuec9D62K1+NwOGhpaaGlpQVYIohFo1EuXbpkGZdIJFKTILaWtOzyoS43HlJK8vk8+XyeaDRKIBCgubm5YTyuMBrGogKV3InVbsRsNsvJkydpa2vjpptusm72DZnkGwjZ2krp9a+HTAY8HtRcDmZnV/2enSAG+hM/Ho8vI4hFIhGam5utlPFGeUuVhnphYQGn04nb7SafzwNLFbUNFbHNRcNY2LBW3YnZ2Vn6+/s5ePAgkUik7LONMhYbanRcLgiF9L9t2hdrgdvtrkoQm5yctAhi2WyWdDqNx+PZ8IlrZlrsnoep5WEaqUY5/uagYSyoX+7OhKZpXLhwgWw2y/Hjx6tmEOqd5FJKxsfHSafTtLa2bkrtx2aiGkHs5MmTTE5OMjAwsOEEsWqZloYQ0JXBz72xWAt3AiCVSnH69Gm2bNnC/v37a25vZ3DWQqlU4vTp0zidTkKhEHNzc5Zr39LSYnkr19JyZjX4fD7cbjf79+/H6XRa7NJLly6RTqctglg4HMbn86154tYqzjPRMB6bh59rY6FpGoVCoa4gJuiT+8SJExw+fJiQ6c7XgBBixQBnMpnk9OnT7Nixg46ODorFIp2dnQBWIG98fJxEImF5Hy0tLeuaYNWwmQbIfj2bmppoamqip6fH4p/EYjEGBgbI5XJL7NJIBG8dzNK1xkMaKmIbh59LY2EuO4aHh5FSsnXr1hW3V1WVc+fOUSwWufnmm+u6qWstQ8yJPz4+znXXXUcgEFjWM8Tj8dDV1UVXVxfJZNLSuBgcHCSbzRIIBCzPw+PxrOHMl49xM1DLEAkhCAQCBAIB+vr60DSNVCpFLBbj/PnzFAoFgsGgZTyqLe9W8yxWw0oqYpqmMTs7S19fn8VAbRiPJfzcGQs7d0JRFCtOUQupVIpTp07R29tLJpOp+0atZixKpRJnz55FCGGVp9ezH6fTSW9vr0XfXlxcJBqNcvbsWUqlksXAjEQi6yJRbQbqmWCKohAMBgkGg2zbtg1N0yyC2MTEBKqqlrFLXS7XZRuLauM0jUc+n2dhYYHu7m5LNxUoI4j9PBuPa+POukKoXHaspH0ppWRiYoLR0VGOHDlCc3Mzs7OzdXMnKo1FKpXi5MmTbN26ld7e3rrHXLkfIYQ1wbZv315GohodHQV0la2WlhaCwWDdJKqNxHpTpysRxEZHRy2+RSwWo6WlZcMNo6qqFm/D7nnYhYDg51eC8OfCWNTiTiiKsmwJALoHcObMGRRF4eabb7ZuyrWkMe0BzsnJSYaHhy2jU4nLudkqSVTFYpF4PF41WNrc3HxFbuyN4llUI4g9+uijxONxRkdHEUJYHtVGGMZqXktDRWwJz3hjsRJ3oppnYQYet2/fTnd3d9lna1HhFkJY2Q5VVcuMzlqwVp6Fy+Wivb2ddqOi1B4sTaVS+Hw+IpGIpb25WdiMCWNOyD179iCEqGoYKwlia4HpWayEn2cVsWessaiHO2Gf/FJKRkdHmZyc5Prrr8fv9y/bZz3pUBPFYpGRkRF27NhBX1/fZdGfL2dS24Oldq2LyclJMpkMpVLJkui7nGDplYR5LasZxlgsxtTUFBcuXFizgth64iH1qIhduHABp9PJjTfeuKZ9X2t4RhqLelsFmpO/WCxy+vRpPB7PioHH1dKhJqanp5mYmKC7u3vVTMuVhF3roqmpiYWFBTo7O4nFYtd0sNSOlQxnNYJYPB5nbGyMxcVFy6uKRCL4/f5l98VGBE+rqYj953/+J01NTQ1jca1hLZRtRVHIZrM88sgj7Nq1y7rJVtp+pZtV0zTOnz9PPp9n+/btGxK138waE3uwdNu2baiqSjKZJBqNlgVLzaKxqxEsvRyYCmKmV2USxIaHh0mn0/j9fst4+Hw+VFXdFPHjdDptcWieznjGGAspJblczoqUr/ajSymZnJwkkUhwyy231KWLuZJnkclkOHnyJFu2bOHAgQNMTEyURdCfDnA4HNbkgaVg6fz8PIODg2Wq4lcqWFqJy1nOrUQQy2azOJ1OPB4PuVyuLi5NvTClDZ/ueEYYC5M7kU6nGRsbs/pv1EI+n+fUqVN4vV6rZqEe1ApwmgVlhw4dIhwOW9tec4Vka0StmEBlsHSzhIE3E5UEMSklQ0NDpNPpugli9cL0Yp7ueNobCzt3wuFwrBpTWFhY4Pz58+zdu5dAIMC5c+fqPlblxNU0jf7+flKp1LKCsmutRH0jYI8JmG59NBplaGjIYpZuVhe1zYYQArfbTVNTE11dXXURxOpFKpVqeBZXE9W4EysZCyklAwMDxGIxjh07htfrJZ/Pr+nGtnsWuVyOEydO0N7ebgWu5hIZIgEvToey5qrTubk5i1Ngl+K7Vo2O3a23M0tjsRi5XI5HH3103ZPrasEe4KyHIGaP56wUDG4sQ64ianUodzgcVUlWuVyOkydP0tLSwvHjx63t19q93Nx+bm6OixcvcuDAAYswFF3MMh1PM5tI0xZsQrL6JFdVlTNnziCEYPv27cTjcS5evEg+n7cm2ma5rxttgOzB0tnZWW688cYND5aaqlmbhZWyIdUIYqbxuHTpEkKImueXTqerkvGebnjaGYtqrQJNVJv85sTev3//sljGWo0F6GxMTdMs+TwATZPMJDL63xJmExmS8SROLcdOTaIoy9fy6XSakydP0tfXZ9UiNDc3WwVW9qdYJpNhcHCQlpaWDdW72MwYQ2WwtFQqEYvFLitYajZt3izUQ8oy4XQ6aW1tte6pymCwef7JZHLNMQshxD7gX21v7QT+UEr5V7ZtXgg8AFwy3rpfSvm/6z7IOvC0MRb1cCfsLvtK8QQTazEW+XyeiYkJAoEAx44dKzv+fDJDSS3fjwbML+a4MBmlPeijtXmptNwMiJql7pVjUBTFmmi9vb2cPn3aemL39/fj8XgsCnc1vsC1CKfTednB0o2U66uGy+FZVAaDC4UCsViMT3ziE5w5c4a3vvWtvOQlL+E3f/M3V820SCkvAEcBhBAOYAL4apVN/1NK+ap1DXgdeFoYi3q5E+b7mUyGU6dO0dHRwY033rjq9qvBDIq2tbURDAbLvlcsqcwnM8v3je5xlFSNqVia+WSWtqCXhanxFQ1YtTEKIcpuRDOwaPIFmpubn3YszHqDpfZz2uiK00ps5P7dbjednZ385V/+JY899hif/exn+fGPf7yerMovAINSypENGdhl4Jo2FvW2CrSjWCzy5JNPlqUxL+f4Q0NDLCwscOzYMebm5pZ5ATOJDKrUjQOAuaJWFAX76jqdzXHi5CnaWsIcO3xo2U2zlqemz+ejp6fH4gtUlqybVadrke6/mlgpWGqekxkP2UysZRmyFkgp6evr461vfet6vn4X8MUan90qhDgBTALvk1KeWe8Y68E1ayyklMTjcVKpFO3t7atOJFVVuXDhAsVikWc/+9mXHX0vFAqcPHmSYDDITTfdZJUt2/UvcoUS8ZQu12YZCUCa/2csiZLJJJcuDbFt2zbC4TBjC4vML2bpCDURbNKfmit5P6s17aksWTel+4eGhsoCc1eLSLVWVDJLzRjO3Nwci4uLPPbYY9aSZSOZpZvhuVxOQFYI4QZeDfxelY+fALZJKVNCiFcA/wbsWffB6sA1aSxM7kQmkyEaja7aJ9QMFvb09ODz+S7bUJhPtL1791quPyyfuNPxNJW3ggYoQg/waarK1NQU8/Pz7N+/H6/HgzR8kNyp04w99RQ+p4P2Fz0X98F9lzVmEw6HoyzwVigUrKrTxcVFmpqarBaMTxeYMRyPx0OhUGD//v2XHSyths1c5qxzTL8IPCGlnKn8QEqZtP39TSHE3woh2qSU85cxzBVxTRmLyiCm0+msmgq1Y2JigpGREQ4fPkwwGGR8fPyyjj88PGyl/ipbD9oDoovZPIuZPFTcBALQpEBVNRLJBA6HwuHDh3RDY24zMID45rcgHCanaYz+4xdxvfF1dBzZT5On3NBdLs/C7XaXxQZMAzw9PU0ul6NYLFrB0o3gQlwJbc9awdKJiQkWFxfxer3WOa2FWboZy5DL3OebqLEEEUJsAWaklFIIcTO6U7uw3gPVg2vGWFTjTtTiTYCeijt37hxSymUCNet5QhSLRU6dOoXP5+P48eNVv2+ncM/EMyAEitDTpRYEZNIZhoaGcLs97Ni5CyQ4BKjGduLcOWgOgN+PAGShQObMeYY6ugj63HSEmvC6l85no2CvOvX5fCQSCVpbW4lGo4yNjSGltJ7Q603RbmbGotbvWi1YanawrxUsXcv+LweZTKbucgI7hBBNwEuBd9re+3UAKeWngNcB/0MIUQKywF1yk9l714SxqNUqsBYjc3FxkVOnTrF161Z6enqqci3W8qPH43HOnDnD7t27V6wONA1RLJUjW9CLxDRbcBOh9zudGJ9ga18f8wsLxvdMQyFRhEC6XYhiyfBCgFIJjIBnMlsgmS0QanLTGfLjdGxu1anJUoQlLoSZonW73Va8o94U7WZ7Fqv9rtUKxlKpFNFolHPnzlEoFAiFQlYA2O5NbYahS6VS6yLWSSkzQGvFe5+y/f1/gf972QNcA66qsagld2eiUvaumjJ2JUxvpB4tBtObOXfuHDfccMOqTwBFUVA1jZl4unw/gKapjI2OksvlOHjokF5xOl+5fBS6cTh2DMfQEHJqSn/b50M9dMgyOoqARMY0Gh6KpZWXYhuFSve+sqTbVBVf7Qm9mZ7FWvcthKC5uZnm5uayYKmdtm0GSzfDWDxTqN5wFY1FPdwJ+zKkWCxy5swZnE7nigI19RKtTMk7KSU33nhjXfwEIQTRxRxBX/n+C/k8AwP9hMJh9m7bjiIEmlpCGuHMZc/atnbUu94EQ0MIRUHduQN8TcYxQJMSEEgJsVSOsXiObdEUHaEmnI4r163M1IPo7u4ue0LbhXLMJ7RpnK/GMmQtsBPeYKnx8/z8PJlMhieffHJDy/DX61lci7jixqIeuTsTprFIJBKcOXOGHTt20NXVteL+a4nw2mFv8GOXfF8NmoSFdJ4wetYDIJlIMHTpEjt2bCcUCuvnqJ8ZUtMQCORycwGRCPLYMQyzgCgY1wOsjAksPaXj6RzxdI7WZh9tQR+OK9zisPIJbf4uJjnMnISbyYXYDENkb/ycSCQ4ePDghgRLTTSMxTphStjV2ypQCEEul+PcuXMcPXq0rkDRapWnExMTjI2NWcsYs9ajHswv5tA0Dc3Y1/TkJPPRKIcOHsRVycwzsh8a0giC6mYBKPM2NE1jeHiYeCyG1+MhGA4RbA7itXcek0vfn0tmiaZytAV9tAZ8VetO6sHlxhYqC6vMFO3U1JQVUzKf0BvVRW2zGZywcrA0k8lYbNl6O6iZy7dnAq6YsdA0jbGxMdra2upq1FIoFKxlws0331z3TVJrGVKrwU+9y5Z8sUQiW0DTJKpaYmBgALfbw+HDB1GEoZQtyj0CaaRJzGyJInQKuG4zBIVCgf7+i4TDYQ4fPkypVCSeSDI5NUUum6WpqYlgKGRM7KV9q5pkJp4hltI9jZaAd12TcSOf0maKNhKJUCqV2LlzZ1mbwmAweNkp2s1c4lQznisFS02BHLtmabXzMg3MMwGbbizsy47x8fG6hGBNUtSePXvI5XJreppUS7eaXcX6+vqWNfip11hMx9KAoFgocOb0Gbp7emhra0PBWJJUpFEdDqVi+aEHN4XQb8JkMsnQ0BDbtm0nFAqhloq43B4rwCilJJvJEI/HyBfynDt7luZgkFAwqCtVKwqFkmrVnbSHmoj4PVedbGVmLMwUbW9vryUkU5miNbUu6v19N9OzqMcQrRQsNc/LpNqbzNJ0Or3e1OkwsAioQElKeVPF5wL4OPAKIAP8NynlE2s+0BqwqcaikjuxEm/C3H5oaIj5+XmLFDUwMLCmY1ZO/tUa/NRjLNK5IqlsgYRBPz985Ii+fkVfZphP/aU0qtSXDlrl00oPeU5PTzM7M8uB/ftxezy6TkPFlkIImvx+/AE/iUSCffv2kkguEo3FGB0bw+N2EQyFCAZDSI+HyWiK+WSGjmAT4cDG6UeuFdUmnV1IBpaXq7tcLsvrWEmyfzONxXrEelcKlg4ODvL444/z0EMPsW/fPkql0nrU0l+0AiPzF9Hp3XuAZwGfNP67adg0Y1GtQ/lKxiKfz3Py5ElCoVBNUlQ9MGMWZjPj1Rr81GMsJqNJBoaGyOdzNDcHKp4U5Te2BBQEQhEgK/crGRwcRNMkhw4dQjGWQg4BxSqTTAiQ0tinw0GLcWMKIJfPk0wkGBsbo1Ao4Pf7CQWDZLJBAouesrqTK4l6YiGVKdpcLmeJ5JgSdNVStJudablc9qY9WArQ09PDqVOn+NnPfsaxY8f46Ec/ym233bYRwwW4A/i8QcT6mRAiLIToklJObdQBKrFpxsJUtrZPeqfTWVXxen5+ngsXLrBv3z7rQptYKyNTURQymQyPPPIIPT09qzb4Wc1YTM3HeexJXWWrt6eHgcFBkKAo5UFLa7wseRvSRtjK5/NcuHiBtrZ2tmzZYhuTRDX2UxkILXc3TLq4RCDweDy0d3TQ3tGB1DRS6TSpxSTTM3oZQTAYZEt7Czu7O2i+wkZjrRPa6/XS3d29aoq2VCpdlnDuStgMr6W7u5vOzk5e97rXcfvtt69VaEkC3xVCSODTUsrPVHzeA4zZXo8b7z39jEU1detKz0LTNAYGBkgkEmXKU9W+U+8PmUqlWFhY4IYbbiAUCtU1zlo/4vz8PD/62VNs3baDYDBIqVRCSg3E0hN02QQHQGBLZJBMJBgeHmL7jp0Eg+VjUoSwaODmMkYI/XvVHtJCGMQuw2BJqf8RbA7Q3Bygq7uHUqnEYjLJ6MQ05/sHCQWa2N3bSc+WjmX1LhuNy336r5SinZmZweFwkMvlLB7ERk3wzegZAnqA02S/rtFzeY6UclII0QF8TwhxXkr5Y9vn1S7yM4fubTcW2WyWkydP0tbWxk033bQq12K1CLrZ4GdxcZGtW7fWZSigurEwYydDE9Ps3X8Al0t/milC9xZ0A7EUpzC9Aj32sHQeEr03STS6wP79B3F73BXGRVIZ1jC1Ox1CIIUwti3bQIctmCqsXKx+bKfTSaSlhUhLCwLIZLOMzCcZHJvG75R0tLWUdQrfSGz0UsGeonU6nbjdbhwOB5OTk2U8iMtN0W7EMqQa1ps6lVJOGv+dFUJ8FbgZsBuLcaDP9roXXddi07BpxqLaj2ZWkZq1BwcPHrSCQ7VQj7y/vcFPKBSiUCjUPc5KY2EWlHl9Prq27TGcfuuk0FR12QQHwysQQs+OaBJN08jnC+SyWQ4fOgRCsbYzdmXEQZdzPBWhGEsTWVbUuqxoTd+T9V9heTw2gyWXmJjQidQ0CmqefHKBXCZNPB7fUL2LzY4ruN1u2tra6OzsLFPYMlO0zc3NVrB0LUuWzQqersdYCCH8gCKlXDT+fhlQqa/5NeDdQogvoQc2E5sZr4Ar7FkIIRgfH8fpdNYtK7caI7Oywc/MzAy5XK7uMdmNhcns3LVrF6rLz8JiFliayjprsrqnZ05kiR6f6L94EYdDYdeunWXehg7dexDoxqXy7OzehPmnQIKsIZAjjfStzcvRjEm7LMuiKCgOHxmlCXfAw74921lMJiy9C7/fT2tr67ol+jabC2Hfd6XClqZplmrYxMQEmqYtS2XWwmapZK3Ts+gEvmqcqxP4FynltyuqTr+JnjYdQE+dvn3DBl0DV8xYZDIZRkZG8Pv93HDDDXXfULUyKLUEeeuhe9thGovx8XHGxsa4/vrrcbq99E9Gl20rjZSoxa2wPpBohkHQC5RG2LVrN0ODgwbd20A+j5ieRjocKN1daIrDvqrArEqt5rmYsQqlIp4hKseCbrQUI8C6HFJfQklI5ooML6RpCQTYs68Dp6L35bwcib6rWRuiKAqhUIhQKMSOHTuWpTKdTqflRVWmaK8lz0JKOQRcX+V9e9WpBN512QNcA67IMmRqaopLly7R19dnUb3rRTVjUdngp1qJ+lowPj5OU1MTx48fx+l0MjqXrOo/CJ1RZT3FhWKSkASqplPJk4kEBw4ctIyXhrHkSCQQn/88RGM4pIa2eze84Y1gxGIkekwEab4qX0qYBkQzPheiljGwBmt4L6ahkdY52L8nJSws5oin8rQ0e2kLNrE1EGDr1q3LJPpM+fvVStavlGexGipTmWYvXDNF6/f7LeNxLRmLaxWb6lmYXIdSqcTx48dJJpPMLyvbXhmVxsJMs9ob/Ky0/UowvZ3m5maOHDmCEIJMvkgikzcZ2baJJcsnp+2zUqnExf4BvF4v+w8c0G+6iqWE8oN/R0kkkFs6dUPT3484eQJ5rIyYpxuXJWoXYBiRMphpWWPyVARWBUtLGd0LWfq0MqZhnpsmpVV30trso63Zt0yiL5/Ps7CwUKYqbk42yzhuYuvCy53QXq+Xrq4uq6t6KpUiFotx/vx50uk0Pp+PpqamuljG9UJV1U1L915pbJqxyGQyPPbYY/T29tLb22uljtbaWdyc/FLKVdOsUL9nYTYf6u7uRlEU64k1FUsBSw94c9oKxDJ3HyCXzXLxYj/d3d10dLRbrM1lT8D5eTS/H4dZhepyIaIxJLqxmZ2ZocnfRMAf0HOiUDHBK2GrO5HC4H1gHHt52rXcc6GMim5f+qiaZDaRIbqoU8jtdScej6eMD2HGB06fPo2maRsmzVcLG/n0t6dot27dyujoKPl8nmQyycjICEIIqxAuGAyu+7ibLF51RbFpxsLr9XLkyJEyF2wtT30TiqKQy+UsRefKBj/Vtl+NUm4anePHjxOLxVhcXAQgkc6TyZcbM/05LKxScjuiCwuMjY2xZ88emvx+fcIJgSIksjIYuX074qc/Rfp8ukEpFqG3h2wmTX//AC0tERYWooyMjOL1eo21dxCvx6vXk1AeWi2LbdhrUgQWyasSmtSs9wVm1a+00sB2lDTJdLx23YkQ5ariZnxgcnKSRCJBNpu1vI6N6rC+mfEQgObmZrZs2QLoWbFYLMb09DQXL15cV6m6lDUeHE9TbJqxcDgcy9Zq9QjwViKXyzE1NcV11123rP1grePW8ixMef9QKGQZHdMTkVIybXgV1WD87FZ6cmx0lEwmzaHDh5e7rNKckEvQnv98HLE48rzRtf0FL2ShcwsjF/vZu3sXLo9ezyGlJJfLkUgkuHRpGLVUIl8okEgmaG4OoCh6hWu1CQ7lvA+LzyElyvQUZLNoHZ0QCBjNCgzynKYty9iY3klR1ZbqTkJ+wv7qHp0ZHzB1Pru7u8uaBjU3N9Pa2npZ3sdm14bYA7gul4uOjg46OjrKUrRmqXowGLQ8j5WWGc8UQwFXgZRV7zLEXlTW29tbl6GA2suQRCLB6dOn2bNnT1lrAXP7hcUsBVXTJ4lZHsryJ7omJefPn8Pv97Nv/4GqN4NmzlGWljHS40Z9/esgn0cKwcTcHMmpKQ4dOoTL6aRYKuneAtLiRXRt2YKqqZw5fYZ4PM7Y2Dhul4tQKEgoFMbtKX/a23kYOhtUD3K6HvgayqOP6Msbj4fC298OPq+ectXksu9WQ6GkMb6wyMJihvZg7boT8+lvb4RkpjQXDE8MWJeLf7UyLSulaM0lWLUUbalU2nT9jSuJK0rKqncZYm/ws3v3blKp2k/81Y4hpWRsbIyJiYmqOpuKolAsqcwajY3tshTSKu7Q38ik02SzOfp6+2hta0Vqy9OognI6hMl3UIz3Sy4ngwODuNxu9h84gENRrGCk6QkowlZApjhwOB1s3boNgHwuTyIRZ3RslGKhqBeQhcM6oaoitSkBZWgIx8M/RXZ0IR0OSCZx3fsV+OVf1gdvjHqJQq4nY6t5LkJAtlBidH4RnztLZ6iJgG/1zmr2lCYsd/F9Pl8ZC7MWNtOzWGv90Wop2nA4zMDAwJrL04UQfcDngS3ot9ZnpJQfr9jmhVzhpshwhT2LeoKPlQ1+5ufn18WbAN21PHPmzDLBm8rtFxZztDbbshfmut5UoZJ6QHRqctJqSmx6+ObZLE3wJX9CKZWQmTSiyY8mIJfL099/kS1bttDe3mFEIY2shW2S6XwKI6ha8bT3+bx4vJ10dOpszFQqRTKZYHJiAofTSTAYJBwKWUpbSjKJdLiRDiO06vcj5heMAGiFQRB6XENQ3dMQtmuTLZQYnkvi97joDDXR5HVZ57Eaqrn4CwsLXLx4kXw+bxWOVWYlNttYrJeUVZmizefzDA8P88lPfpIzZ85w1113cdddd3HnnXfWs7sS8D+llE8IIZqBx4UQ35NSnq3Ybt1NkYUQB9BJXbuklJoQQgG+Dfw/KeXna31vU42FEOUy9iu5kFJWb/Cz1qCoWcBmdimrJnhjh6pJYuk81RY5UhoKXyMj5AsFDh0+xNmz56o+PU3CFIb2pvL9H7D9gX/D2RxEHD5I9KW3MTw1yc6du2huDlgFY1oVurcQS7UnZnDTOo6t94BQFJqDQZqDzfT2CoqlAqmLA8RPniTtcODYuZNWj5tWVUUUC0iXCxFdQO7YiagxqU0ehvmxHkjVlgrYKpDOF7k0lyDgddMZarL2US/sLn5fX58lKBONRhkZGUFRlDIuxGYtQzaykMzj8bBv3z4+8pGP8JGPfIQPfOADdVMGDMr2lPH3ohDiHHo1aaWxWDeklOeEEOeBV6HTxv8EuLCSoYBrpG/ISg1+6qkNqba/p556isOHD69aUDa/mNMrSSuIUAqQKxQY6O8nFA6xdft2HIqeF9GkhoPyG0vYsgri9BmU73+PQjiCbG0l9+jjpDM5DrztbbiNlK8VYBRLzYcsVJCmjCSLHmOoMlfMCe45eRr/vfcaY5Fk0rcye/wmkjffTPuPf4RbcUBfH+K1vwQlvW9JualansWxlkYIVJayKdZ1MjyQxWyBxWyBYiZF4DKyp5WCMvb2i6lUinPnzlnGox4NzHqxGV6Lqc1x+PDhdX1fCLEduAF4uMrHl9sU+WPA7wghXMBzgBev9oWrbixM5e5du3ZVbfCzFvq2SQEvFovceuutq5JhsvkiyVwR1YhI2oObycVFBocG2bZtu6XwpBmScdWpk7a04tgYmser9/9IxlCamugt5NFs3BAz9WkyMs3shUOxPcHTaZSZadxz8walfKny1T4CKYFCAedXv4oMh5EeN6gqzT/7L1y33oJ8zWsovfzlxOfniReLZMbGURQFt9E71OPxIFfwHoSRji2jrmMYSK3c9Ulmi0STRZoiKdqDPtzOy6u3sLdfTKVSbN++3dLALBaLG9YxfjNqQy5H2VsIEQDuA94jbX1NDVx2U2Qp5XeFEH8BfBh4gZSyuNp3rugyxIT53ujoKFNTUysqd9e7DDGVtswgWT2sual4GiGWYhxmcHN2doap6Rn27dtf9vRShEAoDlRN4jCWHeaztmzytrZALkfRoeBvDhIsZdDa2u1bWClOACEUq/rUys1PTuH+5N9CPMH2XBbn/BzF1/wSmhFH0Y2GbYLnsqBp4DFrZBxoTicioxfDOX0+In19tAjdqE5PTZNKpRi6dAlVLelszHCYJr+u71kNNlqHEQSu1BldCgrHUnrrgpaATiF3bUC/EyEEgUCAwAp09Fq1H6ths1oXrofqbTzt7wP+WUp5f+XncuOaIj8EPFlvteoV9ywcDgf5fJ7z58/jcrk4fvz4iha9HmNhBkVNpa2ZmWVNp5chmcmTzhWNGIduLDRNY/jSJUqqypHDh/TPrG9IqwzdFMDRa0RM2qT1PyT37CUbDuMfH8cvFLTWVrSX34ZVs1qjrkMRwloMef7us4iz58DpxJvPoXzt6yiHj6Dt2IFYXETz+xFezxJ9O9CMbGtDLESRkRa09CLC6UIzpOv0CW6EKIXuVTQrClu2bEFVVVKLSeajMVKjo3jcHkKhEMFQCK/XU5UNukQ3L3e0pNQserpZd2KqkLcGfDidGzchq9HR7fJ81ejotbAZxmKd5ekC+HvgnJTyL2tss1FNkQ8Cn6t346vSZOixxx5j165dqzYMgpWNhZTS8k4qu56vlJOXUlotCBUh0DRJIZ/nYn8/rS0tbOnqssq7TZ6EOcGVioluLwMXAqanp5mZmWX/e97D+KOP4NrShXv7NvB4jcyJ1CtCl4/KqguRUiJOnQKvB1xuVAGuRALl4Z/h+tKXIJsFRUF985spHT6spzwdCoW778b1pS/iGBtDa2mlcNddYLjB0va/ZZcllcL9w/+gLRanZe9eSseP6ySweIKxUT2w29zcTCgUtkhh+nXDSrcuaXOUZ0xMaBLmK+pO1tvvZCV4PJ5ltR8LCwvLuBDVFMU3y1isYxnyHOCXgVNCiKeM934f2ApW5elGNUU+BJyud+NNX4aYkFKvykylUhw9enSZ1mYtrNQH5PTp07hcrmV9RczlTy1jEU3lyBV1A+RwOCgWCpw7f46dO3bQXCF7J439WX3FhLCNZ6mAy2wWVCoWOXL4MIoiKG7ZgrZNNxTW2GycBluphi6vZ2Y6VBXp9kCxiHC6EJpEaCqOnz2M7OiA9jbI5XD88z+j/q//hQyF9H22RFDf9S6KqgrKcm/N0r3QR4xSKOD+m/+LY2oazetFefxxWFhAvOIVeLd42bKlE1VVSadTxOJxJiZ0LZJg0KCie31lVawK5jKkyqQT5XUnbUG97mQzjAaU137Y6egmF8Le9NlcAm90piWVStVNJjQhpfwJ1bj65dtcdlNkg88Rl1LWTWK6Ip6FvcFPe3v7mkRVqv2AqVSKkydPsm3bNnp6epZ9vlIndVXTbAQsydT0NMVSkeuuO4LL5apemi6XUpwOZSlCoRjFZcVikYsXLxAOh9mxYwcOxagKLTMsGAbHzHsuPe2FgHyhgNPh1C2I04l6yy04HvopZDOIQhG1cwuKx40WCOhZFI8XuZhCLCwgrYyPkd9QHFW7oNlNrpQC9+gozukZtC2d+hZqCee//zvqy19uFLNJFIeD5mCI5mAIxeCKJJNJJiYmyeVyBAIBQsEgwVAIxelAk3p9iv1cK5cqZt3JQipLe/DK9Dup5ELY6dvZbJZ8Ps/MzAwtLS0bVgyXyWTYtm3bhuxroyGlHAN2ruU7m24sKhv8mKIq68X09DRDQ0M1+4DAyp3U55NZSqqGpqoMDg2hKAKv14vb7bYCnGXLDOyTTOiehapZJKx0OsXAwABbt27T031ySVdTCH3BYZqXGswGpqammJqaRgiJz9dEOBwm9Ja30ORyIgcHSWsS76+8A8+X/hWZzepLGrWo79fI1JhYYoOyrNR9GTQNVTGNSXl9iH4dli8nfF4PXm8Hbe3tlqp4MplkanpKZ5sqilFUp+t8mCTYamNQ66w72QxU0tEfeeQRMpkM4+PjwPro6JVY5zLkmsWmGovZ2VkuXLhQNrHXU3kKupt/8eJFstksx48fX9H611q6FEsqc8kMuVyOixcv0tnZQWfnFk6ceGopZVixBq+EEDo9W5qszqlJ9u7Zi89wZRVb6tMUndGf78sbCWmaxsiwHlA9ePAAAkE2myGRSNCfSMALXkDwFa8klkyyf99+ir/8y7j+6Z8Q6TQSifpLv4S0aXpUsi6Xshd6ULNyWVvo6YZUGmVgEFwufTnzqlctyXFVgdnnVQi9l4np6vf19pIvFBgbGyWZTBKLxWhq8hE2aNFOp7NKLEO/MmbdiW40rny/E0VRcDgc7Nixw2qWXa3i1L5kqQfPJOEb2GRj0dzcvKzBz3oqTzVN47HHHqOtrY19+/at6rLWMkgz8TSxWJzRkRF27NxBc7Pe8VupWGNL22SvTAWY2ZDh4WGyuRwHDx3C4XBaE9U+We1fF/0DiB//WGdnPv8FFLq7GX70EZrDETqPHKGkqkhNo0mTNIUjdPf0UioUiCeTFBYWOHfuHP6mJiLvfCfBUglHS4u+/JBGSwD0zGn11a4walQoM4KBxx4DpxOtrQ2RzSI0FfX666zUaGXIzP6eSW0XwlxuSVwul+UZRSItS4ZvYFAPMBrLFb/fj8OhLKs/yRVVxuYX8VapO7mSuhC1Kk5NUeBadPRKNIzFGtDU1LRsybFWzyIajZLJZDhw4MBlVZ5m80VOXxwkHo+z/8AB3G63VW5e9UY06jYkS7EJ/W3J6MgIoXCkzHBpRj5RKEsBPzOz4RgcgL/9WzDTd48/TgHJbilwul3IYzdRuv123P/8/1AuXNTDBy98IfKVr6KtrZW5uTl2796ti7Mk4kylcjA5STidJhQO4fM14VQEskrjo8oJDnogUqDhOX0a2trA69UNyOwsor8fbeeuZXRvEIgq3pY0SWUYBXRS4hoewf3A1/BISeAFL6Br/35UVSWZSDC/sMDIyCXcHp++3AoFcbttnoQorzvpCDfh97g2vS6kFqpVnFbS0SORCK2trcvU0RvGYg2oVXlaT8zCXiuip+5Cq37Hfgz7DVAqlfj3hx5FFU4OHDho3XRSGq0Ga4zdnBim5mYul2VhYZ6Ojg62bt1a7VvWGt3YCVJqyJ/8FOFyQSRMPl+AixdpdjpRjl6vsyAfeRjX1BTK2BiyrQ1FVXH84AfI3j7Uo0cRmornm98g8OSTRJqb6bnzTvJ9W0kmk0xPT5PJ6B3XI+EwzcEgbpdzSSG8SvWo3shZgaYmRCKBNIlnmoaocLNN4yNEOZHMDqtfigTPyAiRf/pHFLcHBYl44gmK73kP7N5t62UiyWRzJJNxXbNDLdEcaCYcDuMP+K30bDpf5NJMgmavi5bA5gVB11JEVo2OHovFdP3VZNLS9fR4POvlWbwcveGxA/g7KeVHKj4XXOGGyCauGilrJZRKJb13h9fL8ePHeeKJJ9ZUH2KniKfTaf7r0SfwNbdavTWXoHMb9IejQTCymJSUERKisSjjY2O0tbYSbG6mWi2JWbdhGhmHGQU1ituy6Qz5QoEWqSE8bjShk76E04lzoB+tWMJx6ZK+B18TDA3B0aO0/uQnOE6dhpYIyuwc7r/5W+T730/H5ARd3/wWSrHI4rNuZva665menkYoCiGz+rSKqpNAL0hLvvRlNP3rl1BmppGaRO3aQunGYzWu6pJRtXsX9oZLAL6f/QypOFDCITQhIJ7A8aMf6gLFGLEbKSzNjs7OLtBUEslF4nG96bPbpTd9DoVC+qTLF4kuZphLFckXS3hcG3vbXo7X4na76ezstPqYmOrof/AHf8CJEyf44z/+Y1796lfzspe9bFVdT+Oe/RvgpehNhB4VQnytouL0ijdENnHFjYXT6SSTydT8fHFxkVOnTrFjxw6LtLWeylNN06yeIq3d23B7fctcdGt5YQQiFaEYyUexNPENfkgymeTQoYNMz8ws1ZIYRkUYojXLn7r6Wl4+7znkfvoTSKcJN/l0t9/vNxb+GrKkIpv8KGdOIwLNaA4HIpFAGRlBBUJnz0EkgvR4dc7GzAyuH/4Hyk8fQgYCSEUh8M1v4Q2GUJ/7XIrFIolkgqnpabLZLAG/n5DLRWhxEcXvR/TpjayKW/sovP/9KP0DeoDzyGGEz7dsqVEtQ2TGbiprQ0xIoV9JRUo01ZZPEssp4igOwuEQkUgYTWIst/Smz/l83mhG7SddKDEwHSfUpDd+vty6ExMbVRdip6P//d//Pc9//vO58847+fGPf8zLXvayVb//yCOPAAwYrQAwGgjdQXnF6R1c4YbIJq6KZ1FrGTI5Ocnw8DDXXXddmfu2nl4g4+PjqKrKrgNHmFvMLxGhFL1YTJ/gS9vrlG/FoDLrXkZJ1RgY6Mfj8bJ//z4UxVGWVZCGe75M8cbar6BYKnEqlWHLr/4anQP9etn7r/wq4pvfhLExQCKf9zzUdBoxMACZDI5cDpxOlPFxkBLV7UaWighsVZZj40inE8XnQRUKouTH8fhjqM99rh6ga2ujrbUNKSW5/n68H/sYMp1CA3LHjlG4/dU43W5kZydqR6f1xDd5H9VVwDHOW/9cMWgddkOSuvlmQufPIeMJQH9qqy96kZU+tZfYl8GocRECvB4PHqPpM1IXBl6IRkmn0pw/f4FgMEiyo5UD27s2ZGmyWfEQKSW33XYbL3/5y+vafmJiApY3O670Gq54Q2QTVyVmUTnxzT6lhUJhWfak1ndqoVgsMjU1ZTQzupGLUzH7gCwuhZ00ZFK+HQ5z3guyuRwXL16gu6uLtvYO62YXQtersGMpTlEuyV8oFpmbG2XX7l2EQmHUm27CDAaq+/bCQlTvGxIKIf/936ElArNzaM0BUDUco6M4Hn2E2AtfSPA734GMXiwme7pRd+7APTqCJnRRG6VYoORdorsvxRoEofvv11W0tm5FFos4T5xgrrePua1byWQyenozHDJiBWaAVlfsoiLDU/a7WYFcLKOR27qV+Dt/ndDjj+uG7oUvRNujF0Q6hG5Tl3l4tpTvUiYKhNSbNzUHgzhdLlRVZWtfH4lkkkx8jkceGbO0PS+HTLUZxmI92Zsa31nm6NWxzabgqixD7BPfbJDc2dnJgQPVNS3rNRbmEiYSiRAOh1lY1AlYyyFsygwSYYjWGtEK4vEEIyPD7Nq126helFYmQOdRyLJ96XvR/zZfLUSjxGJxenp6CIXCZVtrCBAKor3doEcL1GfdgvPLXzZEaQT4fKh9fSgPP0LqtpeR2/tuXMMj+vs33IDI55FPPAkzMyBA83gp/eIv6vwHRJlGhpidhUAAIQGXC6fTRZvTgadrC35/gEQ8ztT0tCUXZyptacb5mNdl+QS3nbs0xH+khrp7N8Ubb1x21ZdUx8sFhWslo3SavTkAvVuby+1mR18329qDSClJJpOW3gVg8SGCwWDdXsdmNUWGtVHIDZGm1ZodX/GGyCauyjLEnPhmw6DVGiTXYyzMrmfXXXcd8XicfKHI4mKOykAkUhrr6aUJ7rAMgGRycopoNFrWVUxI25JFCErGWKoxEzUpmZgYJ7W4SGdHR/nTbmEB8Y1vIKILsGcvsq9Pf3ru3AEeL4UXvRD3d76rszL9fkRqEcXrBaFQ6tuKc8+epad8UxO53/1dHCdOgKqiHTqk141IUKVWRg7Tdu9COXceWlqgpIIiKHZ0IhAEA36CpQK9Hg+FUJBEcpHJqSmdyu1vIhjUq0+dDsVQ8NK9smqZEZOsVq3eo5IRajJMzX1Ww5I6uW6szKd/h02Ry66FWSwWiUajTE5Ocv78+br7tm6kSpYda10iHT9+HGCPEGIHMAHcBby5YrMr3hDZxFVZhhSLRQYHB4lGoys2DLJ/p1Y2xGR2ZjIZbt6+Hfc3voE2O8vFnfvxHD5aFqdAiLKU6NJAFTRNpb9/AEVROHjwYNnNo9ntjVCQlAy6d7khUtUSgwODuqza/gNMTk4iTKJ3Oo3rL/4cLZEEpxPx3e+Bz4dsb0MJNMPvvh/5ohfBY48j0mnI5ZAuJ7lfeDGUVDSTRi7B4TAIWMEgcts2yGSs9KeusWGs/41xld54F54/+zMYHQGvj9Ib3kBh61aEquL4u7/H8dijCATOXbtwvetdtLW1oWkqmXSGeCKxlGEJhQiHQzT5fFQaCvP31oeom1FFLJX/yyrGxewdawZM7UajMstiqpAHfS587uq3rcvlWpaZWFhYsEoMTD5EKBQq/303YRlSKBTW3NXM2P7dwHfQU6f/IKU8I65yQ2RrfFfqQCZM19Hs3VHPj1QrwFkoFDhx4gQtLS3c0N2N673vhVgMl6rhcP4I8b73IXftsuIUlTEF26DovzhA55ZOtnRuKa8GpVzKTq93kNbNb8Y+8vkcFy5cpKvLEONFnwCqKbXf34+Mx6G9AzExAZqqT/JSCcfJEzjf/Zuov/IrlO75XRyPP4GmlpBHriMdCJAbHkYIQalU0g2w0NuPeb7wBXj0Eb3C1OOm9Ju/ibp1K2VLo0IBzz98DhGPobk94HQid+zAOTpKy9//Pa5Ll1C7upBdXTgGBuCBByi+6S4cDoVAczMBg6ZfLBRIJJNMTk6Sy+UJ+JsMVfGg4cLrAdKlat+l5ZpDiOXSgZgZFesnsDLVUi6Pa0hjQreH6qu1sGcmtm3bZlWdmhkyk8Ld2tp6LZWnI6X8JrpBsL93VRsim9h0Y2FXy0omk5w6dQqXy8W+ffvq3ke1ZYjZB8RUAVe+8AWIx2HrViZzEi2WRHz7W8h3vRvQn3ZLBaNLRiOZTJJMxunt22Z1ozJzHfrav7zUXSCspkQYk2IxmeTSpSG279hJKBSsQhcXoDj0MUiJzOX0DQoFHKOjSKcLUSziuP9+ZMCP+tKXogiYn19gamyM/fv343a70Yw2jqqqopw+Aw8/jGxt1WMui4u4Pv951A/8Qfm1e+IJlIGLaB16kJZkEudnP0vH4CCu6Wl9m+FLaLkscksXYngYM/Zij1O43G7a2troaG9DVfUCskQiwdTkFA6HQtgwHFLTyvQyhBBWrMLOBtXXFhWehnHdFOP3stsXTUoC3tpexWqorDrNZDJEo1EuXrxIKpXC6/USCASIRCIbEr94prE34Qp6FuPj44yNjXH06FFOnDixpu+aS5fKfZX1AcnlwOFgUSosKgJNUSBnJ3/Z9SJ1wzE1NcPs7AyRSIvhWtshjP8XZdF6oRicT2NGzM7OMj09zb79+/F6vGVLE7OQDCTanj04OjrQpqYQCwu6RL/i0Ht5KAoyEkF6vTieOkHxllsYm5hgcTHFwYMHlgRnnE4URVBSNRSjl4o01LyEz4uYm9Mnq/0pubiIFDrBWwL4fDiHLyEXFpAeN6KgX1dlZhY1EEC78UZjUstlMQUrZqEoVgEZ6B7eYjLB5NQkyUQSVdNobW0lGAzicjosw2zvi4ItDlQJk7NimhIzVdvWvHECvXYK99jYGNlsllgsxqVLl6xu8a2tretuvdgwFuuAqqqcPXsWTdM4fvz4urpTOxwOcrkcmqaVdWW370s+97nwwANMJ7MoUuBILSKf/Wzrc8XGq9A0Tdee1HT5vJGxsYoMhxnEM2/ypfSgzr/Qn/AjIyPk8zkOHTqIw2GMRQo9RmJE75XRUZxf/CIym0W98UbEE4/jmJpC6+7BMTuDLBbQwmFkdxdiIUopEGBocAiHw8HevfssiTrT8GjGfuntRTgUFE0FhxORSFDcu0+P7WiaLv7rUBDbt+sB3UJBb8YcT+jaGDMz4PYiNRClgj4j29oo3Xmn9YjXJ+nS8qBaQBfA43Hjbmunta2NS0OXaA4GSafTTBtl62YQ0uv1WufhqDDC1nW3BUKtpaAQ+D0OfJ7L51TUQnNzs0UCzOVyy1oVrrX1YjqdXnODoWsdm24szFRmX1/fugk0JkX80UcfpbOzk23bti3blzxwgPnf/wMyX34Amc0yf/PNdN9yi7V6Nyd+oVCgv/8ikXCEru5uI+ipgKx8zlUG4/T/cSoKmqpy/vx5AoEAe/fqxWTWjS+W3GnX/BzBz3wa6fEgnS4cX3sAGQigbd2qxw48bsTUNKJUQpmbQws0c3H3LpoDfrZ0bik/vpQoQj+ABogdOyi96U04v/wVRLGItmMH8u1vx+ly6dtKFU2VFHfsQHvrW/F++cuQTKIeO0bpxS/G+5vvRuYKCKcTgQu1awv53/t9RLB5WSrT5JFIUc1g2NsH6BEcv99PR3sbEkkhXySeSDAxMUE+n9fFckJBK9YhDO9N92SWYkF2aFLS6vcg1VUFqNcFVVXLguxer9fqFq9pmpWeHR0dLetjUlk4ZofZBuCZhE03Ftddd13VC7qWvpWZTMZSAa9Veappkumtu9F+53fIZ7Msjo5aKVKd4AOLqRSDgwOWvL85wRVFQTWyLYoRCK3eeFiSyxdYiEbZuXOHzpDUT8ZQ0rJD4Dp/Hkc+j2jv0L0ah0PnPEgNZW7OKL6SSLebzKtuZ6AlQvfBAwSD4SpHxlrjW2Xhz30exVtuRRTyyKYmQG+TqLsCCsKlx1fUW28l86xn6WQyRae0T7/v/bT9zd/gSiTQXC7UW29FNPlqMzdtRrCsvWJFFsPkR+heicDj8dDe3ka7IZazmFokmVxkcnISh8NpeR0+n9cwRMuPHWpy4ywWKckr37pQUfR4TDgcZufOnWV9TBYXFwkEAlZ61i4K3FiGrAPV0p4rKVnZYQryjo+P09bWtmKJ+sJilqJBwLITp3Q+j2Bubo6Z6Sn279uHx+vDHsMwGZygG50yT8GGZDLJ6JjJGmwzwqTCTE4sg3A6UaUesFOQurZmXx9icBBUDeF0IFtaKPn9zEyMs/VFL8JvtB3UjLSiCcUWKEYu0a2Fy4lW7ToKPV0sEDidTiMzo6JpEk1VSTodtASDqJ2d4HbjeORRRFsrxde/cRn3weBEWbAzNyvP23wImB6IPU6hKYJwKEQwGAJ6KRYKNq8jR8AfIBzR+7YqNg3RjlATC3OpTas6XQvPwt7HpJoocCQSoampiUQicTk9Q/4MuB0oAIPA26WU8SrbDQOLgAqUpJQ3reuAdeKKZEMqUY+xMPuUKorC4cOHGRkZqbltSdWYS2ZsnoIDzcieSCkZNmILBw4etFxfK+AHFoPTHK+1VkdfsEsJM9PTzM3Ps2vXTmZn58xv6ttqwuohYkJKifNZt5D/1rcpjYyguFy4FAX5pjfh+vrX0MJh8HrJCYGcn6M7EjGe7Eu0X2EYPWF7j7JjYCyjAMuF16EXyUnbtkahnKIyMDBE18QkTiEgENA9o2AQ/utnFF/zWhyKguJYKuOvBbOGxDRkEmEZsUojYpf5M+Fyu2lvb9ergY0aENN4uJxOmoMhtna143Y6rsk+p9VEgWOxGN/61rf46Ec/SjgcpqOjgzvuuIOOjo617Pp7wO9JKUtCiD8Ffg+4p8a2L1pHv5B14ap0JFtNLSubzXLixAl6enro7e0lm82uuP1MPG3xGQQYKkwSVS1x4cJFmpubrdgCoLv+YEX97Z6I/R6X6HqbI0Yw9ODBg+RyubKJaxoJe4PkYklFSImzvQ3XH/8f+NGPycdizG7fxpzHTefO3WwZGaGIRJZUfF4fpRtuXBYP0D2DpRaHyxY6dm+DJUMHLAvYgl6r0n/xIp1bttDa021kfvWorVIsogZbDQ9Es5ZlLoehIlqhJmZncepkMaFr/KJL+FWDGdy0xykwhiARRt9WXb2sUMiTSCRYnJ/m0akRHIaE32Z0DtsoQ+R0Omlvb+dtb3sbiUQCTdNIp9MMDw+vyVhIKb9re/kzdOn/q46rYixWom8vLCxw/vx5Dh06ZLUNXGn7fLFELJWzXpsmo1Qqcub0aXp7+2hZtnxZynIgwaEoFIpFnYBl26poTK5IOMyW7m4ciihrSgTlUnZSSr0WxSziAghH4I478ABdQJeA1PYdjBVyhJ86gRLwknjVq/B3d+E10pWVIzWzMfZScTNWYseS4RDlGwOZbJbBwUG2bdtGsLmZ0s3PwvWd76JMTeqaHi432lvegtvlMmjbEqlplFRNX24JzRiDoqePbYQq85CaBGEYg2U0e5aEjPV4h7ld9SyL2+1h97Y++tqay/RXn3jiCVwul5Xa9BnLtsvBZtC9s9ksR44c4Q1veMPl7uodwL/W+EwC3xVCSODTUsrPXO7BVsJVW4ZUlqmbylhzc3PLKOArGYvpWHrZjRaPxcjn8hw5coSA319mAJbdmAYj0my7Z36YSacZGBigb+tWWiIRY/2teyNmakShXPdCGilLxRC8Nc/dmsRC12oYHh1ly2t+icCv/XdKpRLpeIyJ8UkyuQyBQDORSJhgMIRDUcrGJG37FPb9lp0PVvRRMWIfiUSSkZFh9uzejc9npPOamij+wR+gPPooIptFHjyINNS/FMPbEA4H0kjXarkcymOPQTZDafceij09emoWUcbrMGnpS6lROyGrfKjmJadKlkUIrK7siqLg8XhobdUFjHK5HAsLC5Ymptk8yCJUSYly8iRidhZt716dEr8CNqOQbDUG50te8hKmDVKcHWfOnLlDSvkAgBDi/wNKwD/X2M1zpJSTQogO4HtCiPNSyh9f/uir45rwLMyGQW63m5tuummZla+l1p3J6527TUgprSi11+vVJekBsyLS7AhW6dArRkTPfIJHY1HGRsfYvWcPAX9T+U1uuOnmTY5B/dY01SJr2sdjfgekboAGh9i+fbuhtqW7rh3tHbS3taNJjXQqTTQeY3x8HJfTRdiooPWa3del1FOYZtZhGT9kSUdck5L5hQVmZ6Y5eOAATmcFR6CpCfmCF1jG1E63xrpWoJRKuD/yYZShS0igSRFkf+d3KBw8qJ+wpvdcVRR9yWLXJa1MXVfCHkw1+7dKBKEmD27X0gS2Z8+8Xm+ZjH8ikWBhYYFLly7hcjrZ96Uv0fK97yGMmFj+wx9G/YVfqHp8uDqtC7///e/X+sg0FHcDrwJ+oVa3MSnlpPHfWSHEV4GbgWeWsbDHLNLpNCdOnKjZMAhMcZrl12sqmramfklVGRjQOf+HDh7gxImTSxsahsKkEZcxjdADiSWDwj0xOUEinrCIVpVPQ7OLuln4pHsU+j+hiGXtDQGQkmgsxtTEOHv37ClrtmxfTijCYEYGmxEI8oUc0YWY3n+1VCIY0tWkmpsDwFIdhlmQhVgyUBLJ1NQ0yWSSvfv243Q4lmIG9rFVeC76e7aO8oDyyCO6oTCWhWo2i/sLX0D8xV8aSy+Dhl5SKRaLaJqKpgl9SaUoRoyi4hiUGwpYirMoAjqCFVqgNSZ0pSZm8fHH8X/nO2R8PjTAVSrh+r3fo/CTn+Co0e90M5Yh622KDGDocN6D3t28qqycEMIPKFLKRePvlwH/e73jrQdXdRliFvUcOXKEoBHYqheJdJ5MQV/KZLJZ+vv76enuorW9rUYE38aFqEiNOhTd0xkYGMDhcLD/wAEURdHTnTZX2viq4ZrrN7Dp8TgcipHSrDyqPmkXEwn27T+A0+m0lgdIozBtmX8OCInHrffu3LJlCyVVJZlMMjc7x6VLl2jyNeleh9GTQ3fljYwE+pJO0yR79+61OqRJw1Aslays0OVeWg4RpNN6cEYIFKmhud2wmDKuh8DldKJpJS4OXaK1tRWn02mkoiVSVcuWK+Y+pXmeyyAIVngVoF/remIT3sVFnG43TqNzW6lUQpuf58RDD+EwGgeZNG77vq/0MmQV/F/Ag760APiZlPLXhRDd6CK+rwA6ga8anzuBf5FSfvvyR14bV8WzUBSF6elppJQcP3581Q7XlZBSMh3Xb9Z4PF4mVKOYT9iK71QG5MpKuEslotEovb29VjGZblzQd2RE+vVIvr4k0lS1LLJvxhME6BPr1ClkPM6Ew0Gxu5s9+5ao20tZgOqT1Rix8Zk0JqSDlkiE1tYWI8qeIR6Lcd4QrYmEwwal2kP/wAB+f4Cenm5jP5XXD6vIrdrRrVSouYratxecDshm0dwuSCbRnvsca/tSqaRrnba16lF/aV5jPY6jappeQGYaVoMYhmFgy3+n5V6FPmZZ19Nf27NH30k2i/B6cS0uInfv5sYXvICsQeO2xzpaW1vXRBCsF2YX9/VASrm7xvuT6OXpGDqd1697gOvAFTcWxWKR8XG9we7x48fX9SMtLGbJF1WmJieJxWOWUI0wvQczewBW4LIajVh/aKYYGR3B39RkMxQVxsX0QgxPoFQqUSqVcDgdZdkLc3I7Pv0pHA8/TC6fp9flQvn1X0fbsWPZeZgPbkXRS9nLcge2GW7+rShCf8AjCPj9BPx+ent7KRSLJOJxRsfGSCaTNAcCNDcHkJpeH1LLIBnOBaZhUr73PRz33YdQVdTbbkN97WtBCLRt2yn+1m/h+tw/IDNZtOc8G/W/vR3Qe7QO9F+kq7uHlkjE8kgkZqDUoRPzpDSqdTUj5iORmorToRhxDt0QhP3LvQqoP64ge3rI//mf4/m934OFBeSOHeQ+/nEQYlnLwng8zsLCAplMhhMnTlhMzI2o6Uin0+s2FtcqrugyxJS9a21tRTHSb2uBGZmfjum0bYeilPUBKTMGRiDSjDFQ5VgL8/NMTEywbdt2FhbmbanT6jUKEgmapLWtlfPnzuF0uy0JP6/Xoy8nLlxEPPwISbcbbzCEV1XRPvePFG65tXwMYolnYMYRTJ7EUo0EEI+hnD6t8xyOXg/+wLJIpNvtItAcYHpmmj17diOEQiwWZ2R4BK/PSzisj9FtK4Iydy+N9YDjkYdx/sPfQ1MTmlBw3Hcf0u9H+8Vf1Le76SYKx24yh44EctksAwM6fT5omxhLcV09EiuNzIzicABLGRbd65BIqaIIDQS0BkI1f/t67xf1ec8j8+MfQz4Py6qJddhrPOLxOHv37l2WYWltbSUcDq9riZLNZhuFZOuF2dD4uuuuI5/PMzc3t/qXbDCDnKMzUU6eOk1HRzudnZ2WB1DJGtSfqAYJwgywGR6GlJKxsVEy6QyHDh+iUCgitaVlx3KdzaVApialThbr6SWfzxGLxRkaGkRVVcKhMIHxcQKFPE3GTaY5nch0ClEsoLk9lvmpxi2w6jLMjaancX3oQ3rMAA0iLWgf/BAYwTyTzJRaTDJ0aZhdu3bhN27QsNGUKZvLEo/FGBzsR1Ul4VBIpyT7m8q8IvHIo2gOB4rLrV9TjwflZ/9lGYvyceqp5aGhQXbv2oWvqcra3BZstchi5mtFwSGl0andSLeqKqEmFw6BJUdgPlDMbNiagpCKUtNQVIPP56O3t5fe3l5UVbUyLENDQ7jd7jJeRz3YTF3Pq4VNNxaapnHhwgXS6bTV0LhUKq2536nD4WByeoafPnaaHTt2WgFR3fO3PYmt4jEFVdX04JstoKbL5/Xj8/rYt3+/cTOWytiEJjPRXMZIdMk8ky9gwuPxsmXLFrq6tlAslRgbG2NEUTioOCgmEuD340ilkLt2Iw1DIax4QJXAIktehRACx71f0Q1FOKyL5kRjOL7xIOpbfxlx8iTKqVNkHE7Gdu9k33XX4XEvlyf0eX34unx0dXVTUksk47pMXjqTwe/3E4lECAaDOIJBHKqq64AgUUoFSoHg0jW1jTeeSDA2NsqevXvxeb0sxVeWYCe4Sdu1tb+2thUCxeWipz2it2GUusCPGTxWVXVdzbTXC4fDYXkdoHsJCwsLXLx4kUKhUOZ1VDNg61H2fjrgiuhZeDwe9u7da7mR62mOnM/neezUeQ7sP4C7QrPTFKeVUpeb16SZndDKvA1T+m6LqdNovG8GLSuhSWnswwiuVbkH9HSkxtTkFKVikcMveAGOri2IT30KbX6BeE8Psy+/jebZWSKRCG6Xy2hSXmWC2QKeUkqIxVFcTp3spSh61Wo8jvL97+P83OcolUp4SiWObN9O6fBhfSzFAo5/ewDnU0+itbVTevObwaAaOx1OWtpaaW3Tg3qLqRTxeJzJiQm8e/ew+4dNOBcW9GxNUxPqG9+AUMoDoQvRKFNTU5Z6l9TMICzGuM1rt/xaWY5TlcBuS8Bb1jTIfCprmsb8/Dz5fB5FUSgUCtbnptdxOahnYld6HWasY3BwcEWvY71BUyHEB4FfA0z3+/cNub3K7VZsdbjR2HRj4Xa72b59e9l79fY7Bf1mOXv2LOlcgZ6tO5cZCqii5IRcRuRKGtJ3O3fuItTcXEbWEoawrH1pYGdkWrTk8gcsAKqmMjg4iMfjZfeePXqK8OBBxF9/HKlJgghcuSzxeJyBgQGk1AiFwkQiEfxNTdY+zabCZed1000oFy4g3G5ESUKxSOHoDTj/8R/JO51IrweP14eYmUE8+RTy1ltxfu5zOL//AzR/E8qlYVznz1P86EfB8MSEZaQEwWZdV6KvV1Io5Jn7/d9D/a//Qi0U4diNNEciBDRVL0ATgqnpKWKxGPv378dhsDtNlHE0JFTLs1jnapLyrVoRQXuo+vreJFvdeOONuAytDrvXUSqV9DS38W+tWOvyxuFwWMYBqnsd/f39G8Hb+JiU8s9rfSiEcLB6q8MNxRWJWVQ+SertA5LP53nqqafo7OxEdaUMLUcjPmGodVercNT1NhW93ByYmZlhZnbW8kostW6DrOUwuQ5GILTSUJhEKzN7YC5pisUiF/v76XS7aU8mkROT0NNj4ynoBW0+xUHgW99i66OPQcDP/Gt+iclshmw2R3NzgJZIC81V+lxoL385ajyO8v3vgeJAveuNaM++lfz//QQ0+fF6vUghkEIgikWkpuH44Y8gHEY4FPB6EclFxNmzyFtuwWqQVEFDN/txtO/eA7v3oGoayWSChYV5RoYv6T1EjOuzf/9+BMLIzNTIsggjsyMrnbHySI3J+2gJeHA5lk+umZkZRkdHLUNhwu51mIbDvlSxxzpWw+WyNyu9jvn5ee6//34uXbrE7bffzlve8hbuuuuude9/BdzM6q0ONxRXncFZC6Yg7/79+3H5AhQvzaAaehWalfOz62raYCwbSqUSQ5eGKBWLHDp4EMXhWKbWDXrATZMa5h1uUreF5XUsf4Jmc1kGLvazK5sl9OlP65RnVUV99e3IN7zRIF1JhKri/qM/gpMnIBhCZrO0fepThD78YbRdu1lcXCQWizE6OorX6yYUjhAO68sVhEB785tQ3/QmQH+KXrxwkZ233EL4Zw+jOhxQyIPbhXbwoMW8NPu3gu5lCYeCnJzE+Zd/gRgbQ7Z3WJ3NjbMySFOG5J2iEAlHaIlEUDXJwMAAxUIBoSicPXuOcChES0sLXp+3auGbtBkJ+4OimssvFFFVsXtycpLJyUluuOGGmlIGdm/CFFGujHWsZjg2sorV4XDQ2dnJRz7yEebm5vjYxz5mNT9aB94thHgb8BjwP6WUsYrPq7Ux3NQGyVfFWAghVuyKPjExwejoKDfccAM+n4+Lk1FbdmMJihllp1ziXxhBsuGREVpaWti+fYc+GUyiVY0bXJP6zSPAYj2KKg51LB5jYnyC3bt2E37Pb+nGJxDQ2Ypf+5qeYty5EzQN51/8BeKnP9UnbTKJ2t2tj/3sWejuJhQMEgkFUaUkl8sRi8YYHBzQhVTCYULhCE1NPvL5AgP9/fT09uB517sptnfgeOIJZChE8W2/DG1tkEohA80oFy7oQsChEHLbNrQDB/C87/1oyQQyGIJYDPcf/zGFT/w1BJqXqkE1g2RmLA+klAwODtDk8+kpWQTFUpFEPMH4xDi5bJbm5qCu7B0MLluaQDlHpJJLAhDxe5d5FWNjY8zPz3PDDTfUPZFNY1DN6zD/tn9uNzKbVReye/dudu+uyq9asZAMvTP6H6Hb3D8C/gK9+tSOagGRTY2sXpVlSK3Aj70U2RTkXUhmKRRVK7th7QP78kPYpN4kmUyOaCxGR3u72RJu6VvS1vjXGIdi9AhQVbWCkVl+9SWSmekZorEY+/bvw1UoIrM5K1uhOfTGySK6gNy5E3HuHMqpU7pQrlrSU5PT08gtXeD1mBfDUtL2eX34un10d3dTLBVJxuNMTEyQyWQolUr0Gq0QhSJQ3/IW1Le8xXY9Jc6//jjEosj2dkiloFik+Ju/Cek0MpkAg+GqNTUhs1nE+DgcOFg+wY1rqBn091A4zJbOTutjp8tFW7suqa9JXbAmEY8xNjaGx+MhZEjQeWysXP3hsEQ110AvhFOWxyouXbpEMpnk+uuvv6xJXOl1VBoN0+swYx4biXqo3qsVkpkQQnwWeLDKdle8jeFV8SyqoVAocPLkSSKRCPv26UI1qqYxk0iDEDidFRmLCqKCZgQSkvE4Q8O6R+EP+G2bG96HSXwS5vd0DQqfr4kzZ04bwcewUQS0VDSmaiojIyNomuTggf36AJxOZGsrxONozc2IQl7PHPQYBiqbAUWgdXbCxASUSkhNonZ3I4/fbC0bqgXk3S43rW3tOJwuxsbG6Nm2lXQqzalTp/B5vURadKKVy6kH/VBVHE88iRYJ69L/4TCkUigTE3DjDfq1Kqm6BJ+qopRKyObmqgcvlopcvHCBzi1dtLfpgTxzM8X+txCEgkFCwSACqdOp4ybvRCMUChIJRwgE/JgPwqXlnSDiX4pVSKkvd/J5XVpgIydwteWKaTjS6TRmA6f1BkkrcZl1IQghuuRSS8LXAKerbPYoq7c63FBcE8bCZHbu3r27TFFoLpFZUsASDjRNLSNX2SGlZHp6moWFeQ4ePMDC/HzFUqeSjWl4O4a7vWvXTjRNkkwmmJ2d49KlYYuH4Pf7GRoaIhQMsqWry7YvQfH978f10Y8i4nEEUHjnOxHduqS83LkL6XShqCpaXx8iGkX29VH6o/8NHg9mBFBWjM7kW8zMzrKwsMD+A/txu1zI9nYEkM7oPS4uXuwH0PUcwmFcPh+UVBSn7dyafEi/n9Jb3ozzn//FOkbp5bchenuB8krUXD5Pf/9F+vq2Eg6FyrMc1E41SiHwen10b/HR091NsVgikUgwNzvL0CV98oSN+hWnwwEKdBixCiklFy5cQErJoUOHNrxOww77ciWdTjM+Pm4dc71B0kpsgLL3R4UQR9FvhWHgnQD2QjJDcm9Zq8PLOehquKrGQkrJzMyMxey0X+BCSWVhMWu9dij6MkTKsugEoD8tLl0aAuDgwUP6D604jErFyiXL0rHNDl96abmCokgi4QiRsM6QXEwtMj8/z8DAAD6fF4fTiVoq4nB7liZYby/Fj30MkYgjA83gdlsej2htQf3/fg8+/VmIRtFe9EJKv/pr4DXy8TZDsZT21f8eHRsjm82xb98+HIpicBT08/U3NdHkMyZlqUg8HmdsYgLfL/wCWx98EIeUKE4H7D8A1x/VyWivup3Cvv0oE+PI9g7koYNWWkcxLHAqnWZwcJCdO3cQ8Ffc7MZ1so+77Gewr2Q0idPhoL2tldbWFqSUpFNpYvEYU1N6B7Md3Z0UWptw+HycPXsWt9vN7t27N9VQ2BGLxbhw4QJHjx61JANqBUlNo1Gv4bhcZW8p5S/XeN8qJDNeL2t1uJm4YjGLSiiKYrWOM5mddszE0+WNch0OiqWiEbwEkwhULBS4cPEiLZEWtnRtsQqSFGMZI7VytWnAuiH0tOiSG2x+DzM1C3pXsAMHcDgd5L77XcRX7kUtFsnfcgv86q/gDQQQDge0ti4LL0kJ7NpN8aN/qpd0Ly6CcWNWxnGsEIqUDA4O4XA4raDi0nXEKmc3q0ZdTpcuetvWjrZ7N4tHjlA8dYpFp4PCTTcRjkcJhyM4HE7Enj16Vaa5L7l07ORiktGR4WV6G9b4WLom5nUyy+yrUdet8zd+a7PvaF8vFIsFIm6VCxcuEI/H8fv9llr2lTAWZtvCG264oUyRrVaQ1LxfzL9XI4Rd7jLkWsVV8SxKpZLV6enGG2+seoNEAl7yRZWsoVlhNkc2u6GDIJ1K09/fz7bt24iEQkabPh2KQfzSZfV1CLDk8M2naSVUKVEkLCzMMzk1xd69e/F6PIizZ4n8yxdRfT6k24XrJz9hHsnAS15CKNhMKKw3nbHTwc3lkvKd7+D83D/oBszvp/iBP0BWENUASsUSg4ODBEMhuru6yoPCUKZ7Ya+7MLM8ihAEjh2DY8cII8lmskRjMS5cuIgiWFLd8vp0T+HhR1CmJkkGQ4x2tLNn7z68HvfSxLcb6wrdjbJWC8vOxLYDafApbPyUrrYw7c1eYgsL7Nixg0AgwNzcHBcvXsTv91s9SdcqXVAPotEo/f39ywxFNayWmq1FCHsmNhiCq2AsTGUsn8/H9u3baz5JAl43u7vcxFI5ZuJpS1zGdMWj0SjjY2Ps2bsXf5MPs/GxXaJNVbUyb0IzNTKNasiq9G1gbHycVDrNgQMHcbmceqn3U0/pE8Tt1pcKzc10DA4R/K2DLCYXiS4sMDI8TJMR5wiFQridTrSRYZyf+xzS40V1OmExhefDf0LuU58uO/e80Smta0uXpbFgckmstUqNWakZmQWzbQDoRsTq59nTQ6GoL1dGR8co5HPs+Oa3CD/8MJRKBAQcecUr4ciR8liQNT65XKCnbBu5LDVazXMCfTkZ9rl46qmn2LJli6WO1tbWpi9X0mnm5uasfrim7mYgELhsr8OsKj169OiqhqISK6VmK2MdmUyGbiNF/kzCFV2GmE+PI0eOMDIyUhflOxLwEmry4FDzxKJRpJRMTEyQTCQ4cPBg2fLFpHojTBFZo+BcLomwKIqZgRBLdGPj+6qmcWloCKfLxb59e/UUqrmMafZTNluLRWRzM06H0+pYJaW0unNPTk3idDjpHhqiDcAkFgX8sBBFKRTAqzdSTmcyDA0NsnPHDvz2WIGNsSqrWAszEApLHAlz0ioVDBG3y0VHezudHR2oExN4f/Zf5B0OVLcbpxA4vv0tCq++A0eXrW2izXvRWM6RsFeWLpXZVx+riaDPyamTJ+jr6yvTDzGPYy5XduzYQaFQYGFhgeHhYVKpFKFQiPb2dlpaWtZMpDINxQ033LAhHstKXsd//ud/8sIXvvCyj3Gt4YoYCyklQ0NDLCwsWMpY9VK+QZ8AnZEAqZiX2fFhiiW5JH0nyoOXusamXkimaqrhUeiMTEURVIrV6De4oFAoMDAwQKSlpYxXYKL4ol/A/f0foEzP6NPA46F499uwTwohBH6/H7/fz1alj2w2TzqRoFgooAIOtxt3oYAWCiI9bpCS5GKSkZERdu/ejc9bvfy5rHkPS4dcVpAlKTN+1fcFSjZLSSgIh5Mml9MI9ha5dPYMxUTMMn4+r88yCIIq3ksVypoZT7K8IhtUtcTYwBC7du3UGwutArfbTVdXF11dXZYw79zcnFGL47GWK6uVjc/PzzM0NLRhhqISptFQFIXPfe5zFAoF3v72t2/4ca42roixmJ6eJpfLcezYsTJ3bi2Vp6VSiejCPPv27qVjSxfTsTTJTL5Kj1EdiuJAVXWPwgycCUTVisd0JsPgwADbtm4laOhALEMgQOEjf4ry8MMohTzqkSOInp7q7rnQPRKv14Pv1ltQ7roL1333oeXzFIH+17wG5/AIisNBMplk3779Br2bZQ9kUfb0Nt5TdAZsLaNgFotVVoIKYXhPxQK7/H7cmTQ4HYhMFmdrC7uf/zwKCGKxGCMjoxSLBULBEOFIRJcs1JY4ErXK7O1jtscp8oUCs2ND3Hz9wRXbUNZCpTBvJpNhfn6ec+fOUSwWaWlpob29nVAoVLZcmZvTNUuPHj26KYbCji984Qs88MADPPjgg3XrXjydINZQe1/3hpVQVXXZkmNwcNCKgq+GRCLByZMncTqd3Hrrrdb7i9kCM/G0FQQ1IdCDTBf7++np7iYc0clLmtSW1TKY2gzmk71ar49qZy5ZEtyp5p7bvyMEyOkZlEQcrbsbtcnP8PAwyWQCh8NJk8+3JL7rclZ8t7pOpzB8/mXGqgrLyxSf0TSVixf7CQaDdAPOj30MZWwMuW0rxff8DmzZUuY9qJpGMpEgFo+TSqXKxul2uZD261T78OTzOQb6+3nJrUdpNTQiNhKmhurc3JwuK9jcbMVAxsbGuOGGG5Zl2zYaX/rSl/jCF77Agw8+eLmZkCuTO14Hroix0DTNUj8yMTw8jMvlqin/b2Jqaorh4WEOHTrEhQsXOH78+LJt4ukc07E0RYOHIaWGpunxg3gsRjwex+l00traQiQcwWncONMzMywsLLBvzx7rPRPC5kZXu0T2SWyPfdRiZEqL76ExfGkEIWDbtu0IgZ61iEaJJxI4HArhcITWlggej7eqoTD3ZXr75tO7clx2qJrKxQsXaWtrK1sC2PdV7fyt9CrSEglOJBNWd/GWSESvfrXFYe3IGNJ7zzp6iF29y5d3Gw1pUNCHh4eZn5+nubnZ6qfa1NS0KanZ+++/n89+9rM8+OCDG6G72TAWlcZibGwMKSVbjS5Yyw4mJf39/aRSKa677joUReGRRx7hlltuqXEMyXwyw2w8RVE1yVhL1z2XzxOPx4hGY1YTZIfTyZ49e3BWKYACrH1UdjSvhWVeScW+iiW93iLY3ExXd7dZJV+263yhQCwWIx6LoaolwuEIoXAIv99vnY+o0j5AJ3Utb39o7rO//yI9PT1EwpGy3qm1jJv5QdVGRgJy+QLxeJxYLKYvV0IhWsIR/M0BawypdJqhoSH27tnDDXt69A5rVwBmafvRo0ct8Zz5+Xmy2SyRSIS2tjYikciGULu//vWv89d//dd84xvfsNptXiZ+vo2FlNJSODIxNTVFLpdjRxXV61KpxMmTJwkEAuzZs8e6Yf/rv/6LZz/72TWPoaoqxZLKfDJLNJ0vmwXmBCupKv39F1EUBwLdRQ6FI1Y9iHmjW1lae5zAcM+rTVZ9o6VfuiLuSLFY4OLFfrZ0dtLW1gaYWY7ankuxWCSeSBCPxchmswQs7Ytmi0xW+R3z4OYus7ksg/0DbN1eLqprjrfWr2o/R7uXUc24qJpGIh4nFo+RSafxNTXh9fqIRqPs3bOHrVta6aghbrPRsBuKyqWHpmnEYjHm5uaIxWI0NTVZQdK1plIBvv3tb/Nnf/ZnfOMb37Ak+DYADWNRaSxmZ2dJJBLsMRiFJkxZ9m3bti3LVT/00ENVjYVpKKxAphDkiyWm4xkWM3lrO/0JWz5hNamRiCeIxWKkDJpuSyRCOBxi2e+2RDuoCvtEsi8Pcvkc/f39bN26jZC9mZLNuNhjH2Y8xP7TaFKSSi2SiMWIJZL4fF4ihmq32WSojEQlBElDyHf37l00+ZZPVrObmaiIfdS0IWbhWxVDaQU9kUxPzzA1OYnL7cLjdnPDnj62dHZsetBvenqa8fFxjh49WlMDw4SZ5p6bm2PeqCMyOR3Nzc2rLld+8IMf8Ed/9Ed885vftO6lDULDWFQai4WFBebm5ti/f7/1XjQa5dy5c2Ud1O2oZizsjLpqbmU6V2QmnmZ2IcbQpUvs2LGd5oDxhF02KyTJxUWiCzGSiwl8Pl/5hMTkKZWv5833q13K1GKSS8Mj7Nq9m6YqGo32QKpA30dNz8WGTCZD1IjHOByK0ZIgYvVEjScSjI2Osm/fHlwer5XuXDrT8rvSbtxqnYs5XotLYWxk98KisRiTk5Ps27cXl9NFyOuAgk60WilrcbmYmppiYmKiLkNRDcVikYWFBebn51lcXCQUCtHW1kZra+syTsePf/xjPvCBD/CNb3yDzipp9svEz7exAF0iz45EImFV/IEewzB/7Gq1CVBuLCxGptHWbqUbb2ZmhtMXBmjr2YHD1hy46qQQ+lNdVSWZbJaYEXh0OvSOYKFIxNJqMLMM5cVtS1iIRpmemmL3nt14PV6d2GT3PKrGCpaMRiUUUdGiwJighUKBmBE/KBWLuNxucrkcBwxRXczYi22ctQySMAZXK+4iK7wXDZ0Rq2mS+fl5ZmZn2bd3L06nE4dDYV93i6EXsjxrEQwGLZLVeia4iampKSYnJ7n++usvaz8mpJQWpyMajeJyuWhra6NUKjE1NcU999zDgw8+uFkszWvWWFy1qlNTtNdsFZDP5zl+/HhdzLx6DYWUes/PWCzG82+9WRfTWcwym8jok7yqBLW+JFAUgd/fhL+pid7eXnL5PIl4zNJqMHP+Pq8Xh0OYnfksTE1PkUgk2Ld/H06H03oiWz00zINVwBLUrUKSqOxlYn7d7fbQ0dlBZ0cHU9NTzM3N429q4sKF8/gDzUQiYYLBEE6HAgaRaiX6tlXQZYt9UMW46ePRt5mZnSEajbJv3z69BB1oa/ZZhgKMjvEdHXR0dFgTcn5+Xu9+7nJZWYtaD4tqmJycZGpqiqNHj26YPJ4QwiKmgS7KOzk5yTve8Q4GBwd54xvfyNjY2DOS0r0SrpixqCbaWywWeeKJJ4hEIroQ7Cpuqak5YBKCVjIUmqZx7tw5FEXh6NGj1hKlLdhE2O9lNpkhuphbPgOMp3Vl7MHn9eLt3EJn5xarLHx8fNzoXhUiEo7QHGxGk5LRkVFKpRL79u4rH5/Up5bFxKxAmcK3LS1ruf5VlwZG9kWTTExOks5kOGy0BZBSI5VKE4vFGB8fx+PxEonoyuKWaE7Fzir1MpdSyNUEBkEIhcmpSZKJJPv37QNhku4UWptrxyjsE3L37t1ks1nm5uY4e/YsxWLRih8EqwgZm5iYmGBmZmZDDUU1+Hw+EokEhUKBn/zkJwwODnLu3Dme9axNlby85nDFliGFQqHs5ozFYjz22GNcd911da/7Hn74Ya6//nqrRLjWTVQoFDh16hTt7e309fXV3C5fVJmOp5eCoLUie0IgpERWmbGa1GnI0WiMVCqFpqkEm5vZtm17TZfY3E29HAlzaVKNAGamTIeHhwHYsX07CGWZUI1Eks3miMeixOJxhNB5EpFI2KKZ114aSQRK2TIKQEMyOT5BNpdl167dKEYAFASd4SbaqjQ4rgelUsmKaS0uLlrLFXv8YHx8nNnZWet+2EycPHmSd77zndx33301NTU3ENfsMuSqGIv5+XlLGem5z31ufQeXuppSNBqlvb2djo4OnXtQYQjSaV16bteuXXXVH4AeBJ2Op8kVSquu1cuKz2zGpVgqcuHCRZqbm5FSkkzEaWpqsvqM2gOkVR7oZUZj+fGrE8BM3Y2BAV1Ut6e3R1/GCF3+Tr9u5fsy4x5mFWosFqVYKBIKNROOtJSlj5edvzB1QvTlx9joKMWSys6dO8q+43Qo7LXFKi4H9vjBwsICHo8Hp9NJPp9fk6DvenH27Fne8Y538OUvf7ksGL+JaBiLYrGIquo6lrOzsxw9epTHHnusJm/COmhFfKJUKjE/P8/s7CzZbJbW1lY6OjoIBoOW+tHhw4fXxaRLpPPMxNMUSks1K9VYiUDZmj6Xy9E/0E9fbx/hcBizS1k2myMWixKPxVGMjEVLSwtu1/IaBWF4D2ql91DD2zGvxcX+i0Qi5cVvtVK4tVwHVdNYTCaIxmIsLupaDGaZfXXCml4Y6HA42bpt6zLjsiXiX7dXsRoGBweZmZnB7XavOd25Vly4cIG7776bL37xi1Yg/gqgYSzy+TynT+u6owcP6p3Pa/EmrAOuEshUVZWFhQVmZ2eJGuXr+/bto6OjY93sPCmlFQTVtOoMRkBfmiBZTOksxZ07dxIwagKqfadQyBOLRYnG4qiqplOlW1rwVeu9IewSe9WPXyyVuHjhPF1dXbS0LBVmrWTcKpcR9uMt0bohk15kYUGndXvcHks0x+1yoUnJ0OAgXq+H3t5eY3xL+9pIr6ISo6OjRKNRi9Frpjvn5uYuu4S9EgMDA/zyL/8yn//857n++us36AzqQsNYPP744wQCAbZt22ZN+pWMxVoyHgMDA6TTaXp6epifnycejxMMBuno6Fj3jVNSNWYTaWKpfI2liSAaizIxPsHevXuW6jhqeAJmDYYC5IslEvE40ViMfD5HOBgi3BKpugSoBpPkZYrqlnMkaqQ9jWFVlpnr51J9aQSCTDZDLBYnHo9bQi/hcIitfVut38QkdyGhKxKgNbjx5KuRkRFisZhlKCphL2GPRqN4PB4ru7JWdubIyAh33XUX//AP/8CxY8c26hTqRcNYVPIsYG2MzGpQVZXTp0/T1NRUJvZauc71+Xx0dHTQ1ta25urDfFFlJq6Xw1sQgunpaYvOvBSP0C1CVf5CBa/B9B6kppJIJInGYpbQa3mq07wm+n/TmQxDgwPs2LnL8mSWjqFYdS/LB8DSL2iLPVQyRa3jAXbnoKTqKW69GbJGLp8jGAwRMcvXhcDldLCnK7LhXsXw8DCJRGJNLQLS6bRVE6KqqlVAt5ri1vj4OG94wxv49Kc/fbWyHQ1jUSqVlulXPPTQQ9x6661lP56U0ipnX+nGyOVynDx5kt7e3hXz3aZU28zMDPPz87hcLjo6Otb8xMnki0zF0mTyBcbHxsjl81YGwIT5hK4Uqam1NNC/Y0tXIkmlUkSjMRaTCdwer0E9D+N0OUkmkwwPm6K6y5/eVpq10nuoGfcwr1GVz2zGrVQq0d/fT1tbO23tbXpsxeiHGosZ5etNTezf1s3ubd0bQowyoZfyJzl8+PC6l5bFYtEyHKlUinA4THt7O5FIpMzrnJqa4nWvex1//dd/zfOe97yNOoW1omEsqhmLhx9+mGPHjuF0OtfEyFxcXOT06dPs27dvzQU82WyW2dlZZmdnEUJYmZV66hY0TePhx58iVRJs6e4pWzJUMwim96CsEPdALhepMZHJGqXr5hKgVGLv1j4CM7NItxt27Vqa8cv2bfceajEybQSssnL0pXMxszxdXV20trRUZX5KJPlclpBSIBqN4na7aW9vp62tbU0Eq0pcunSJxcXFyzIUldA0jXg8bhWTeb1eFhYWaGlp4bd+67f48z//c170ohdtyLHWiWvWWFzVviFmg2SHw1G3oTBl1a677rp1iYz4fD62bdvGtm3byOfzzM3Nce7cOUqlEm1tbTVTssViUfdktnTQ29tLdDHHbDJjtVR0VOkqboYwqpMXlmhO0u4BsMTibPL5aOrpwe12Mzs7S6fDgf/Xfx2RSuNEUjpyBO1P/gThdC2ngsvVJfZM+7+UljWWH8bxC0W9M1lvb6+uMQqWXFeZhgeCHd0dFgnLVLE6c+bMmpYAdgwNDZFOpzfUUIDurba0tNDS0mIVk/3Hf/wHn/70pwkGgzz00EMcO3aMoL3grwHgCjM4K2GyOJ1OZ12BzNHRUebn5zl27NiGKB95PHpEv7e313JVBwcHl6VkzSXPjh07rI5prUEf4YCH2USGhcXsciq27byrS9xVedqbT3axtCyZmppmcXGRAwcO4Pv934fkIrjdaJqK48QJpj7zWXKvfIVVur48s2L3HiqIWpWHNximQkhyeV0DY9u2pdJ2ezsAu36pUxG0BJY8iKamJrZu3crWrVut63rp0iXS6bRVSBYOh2saAfM3MJmomwUhBLlcjgceeIBPfvKT3HLLLXzrW9/adFWtpyuu2DKkmrTeqVOn6O3ttZ44K1G3z58/j5SSA4ZQ72bCnpJNJBIUi0V27txJb29v1WMXDCZoWRCU6qEC8+ltZ3aXfW5MSCkl4+NjFIpFduzYiRACz1vejIjFwVxn5/MUX/lKYnffrXMkUin8fj8tLREjQLqcI7GaApiiCNKZDP39/ezauYumgN/qmlBr2na3BGhZgdptwtSTmJ2dJR6PEwgErOWKuRQdHBwkl8ttehtDgHg8zmtf+1ruuece7rzzzk091hrQWIZUQkpJKBTizJkzRCIROjo6qj5tisUip06doqWlpSztuplwOBx0dHTgcDhYXFxk7969JJNJHn74YZqbm+no6CijHrtdDra2B8nki0zH0mTyhipYlZzkUo2YoLJ3ienma1InPbndbnbu3GXRqLU9e1F+9jOjzQHgcsK+vYTCYULhMCBJpdIsRKOMj4/j9egciUg4YgUdLS9HVjm+gMWU3sJw167d+Jt0YpWimAK9y6+Vy+kgEqgvLqEoCq2trVZflMXFRebm5hgdHbUo/E6nkyNHjmz675xMJnnDG97Ae9/73mvJUFzTuGKehV1az9SgMF3kWCzGzMwMiUSCUChk8SNyuRynTp0qc/+vFCYnJ5mYmOD666+3VKGllCSTSWaNhsW1UrLJTJ6ZeIZ8sXpflDKGpS1zoSiCYkmlv98Q1e3qMjZC/0I8jut370GZnARNQ33+81HvuUevWSk7gP4Vs4FyLBbTNS/CEcKRCD6vp5y+bpC10uk0g0ND7NlT0ZbAsCtUWTp1tzaXLUHWAykl58+fZ3Fx0VJ930xmZiqV4vWvfz3vfOc7efObN6fxeC6X4/nPfz75fJ5SqcTrXvc6PvShD9Xz1WvWs7iixqJQKKwYyJRSEo/HrTRnsVhkx44d9PX1bXoNgH0MQ0NDpFIpDh8+XPO4Zkp2dnaW+fl5q/zaTMlKKZcFQaFGrEKAkJKiqnLhwkU6OvTepSbsvVGEpsH0FNLtgfb22sFTsdTWELmk7ZmIxSiqqlFEFqGpSVc0TyYSDI+MsHfvPktAp9qY7ZkTt9PBnu7IZU1mU2u1VCpx4MABTDlBOzMzEolYqc7LXYJmMhne8IY3cPfdd3P33Xdf1r5Wgnl/BAIBisUiz33uc/n4xz9eU0PWhoaxKBaLlrEw29nXwtTUFKOjo+zcuZN4PM7CwgJNTU3WU3wj8/h2mGXtDoeDffv2rWkSmCnZubk5pJSWboPbYwZBc1aatNolLxb1Bs+9Pb11Cb/aWZNVP7MdRNiNDXoaO25oZmazOTweD/lslv0HDuD2uMv3uQJHoytSX6yiFqSUXLx4EU3TakoUmKnO2dlZYrEYfr/finOsNRCZzWZ505vexOtf/3p+7dd+bd3jXisymQzPfe5z+eQnP1kP0athLD7+8Y/zla98hVe96lXceeedVUvHzae6mVu3r7NTqZTlcXi9XuspvlGRa1MkeCNiI2ZKdnZ2lmKxSFtbGy2tbSwWxbIgKOg38cDAANtNyT9b6mSlsnWT3l2ZIq1qkMRSStaO+fl5xicnaPYHSGcyeoA0osdAFLG845uJy/UqTENh1vPUsx/zPjB1Mx0Oh0XpXo0nk8/nectb3sIrX/lKfuM3fuOKxL5UVeXYsWMMDAzwrne9iz/90z+t52sNYyGlZHp6mvvvv5+vfvWrpFIpXvnKV3LHHXewa9cuMpkMFy5cwO/3s3fv3hV/TDsj066+tN6OU/l8nhMnTrB169a6mh6tBWbq0KySbWoOobn9ONx6AVkqlWL40hA7d+9ZptFZa6JCNe9BWE2fq/2mRsjBSK3qy4i5+Tnm5ubZv28viuKwGKSxWJxEIo7X4yUUCRMJ602a7Ohpba47sLlsLFKXGxBCrPpbr4RcLsfc3Jyl71lLMKdQKHD33Xfzwhe+kPe85z1XxFDYEY/Hec1rXsMnPvEJDh8+vNrmDWNRibm5Ob761a9y//33Mz09TTab5d3vfjfveMc71vRjZjIZy/1XFMViZNbLHEylUhYb1GyNt1lQVZVoNMrs7CxT8zEyJYVEKs3+/fuq9jmtRd+utZSp6YWgB0/tpLGZmRli8Rh79uyt2c8jl8uxEF0gbojltET0AGnQ37Rur8IMZjocDqvNw0agUjDHFATu6uriN3/zN7n55pv53d/93StuKEx86EMfwu/38773vW+1TRvGohbOnz/PG97wBm677TbOnz/PxMQEt912G695zWusUvZ6kcvlLCq3lJL29nY6OztruqjRaJSLFy9y+PBhAoFA1W02C1NTUwwNDSE8TYzOxPD6mmhpaSEUCumTtzJWYHgFuvewsrGoJIBVdh2bmJwknU6ze/dunA5lGfO0GvKFPIl4nIVojBafwh4H9D74IJ5CAfV1r0OztZWsBSkl586dw+VylRX+bTTMCtTPf/7z/O3f/i3Nzc0WlyJUq5ftBmNubg6Xy0U4HCabzfKyl72Me+65h1e96lWrfbVhLGphamqKeDzOgQMHAF31+8EHH+S+++5jaGiIl770pdxxxx1lOpr1oFAoWIajVCqVqWuB3mNidHSU66+/fl0NZi4HJhP1uuuuw+l0UlJVLk3MMjCmXwuPx0trS4RQKLwsmGsvR7fDrBK1/5wW/0os9fQYHx8nny+wc+dOI2NibGt4HtbrGl6K2+VgRy5O4LnPRVlcRGgamsdD9JOfxPv619f8jaSUnD17FrfbvamGwoSqqrzrXe+ir6+PN7/5zXzta1/jta99LXv37t3U45o4efIkd999t0UTeMMb3sAf/uEf1vPVhrFYD1KpFN/85je59957OX/+PC9+8Yu54447OH78+JoMR7FYZG5ujpmZGfL5PC6XC03Tqnat2kyYDMWMIapbeQ6Fksp0LMXMQoIFo4DM6XAQthS2XNYSpNJo1JL212s9dL3MkeERJJLt27dTq6N85XKlEr1tzbR/5I9xfvzjCFthYHb7dh76u7+rqpdpGgqPx8OuXbs23VBomsZ73vMeIpEIf/qnf7rpjN8NRsNYXC6y2Szf+c53uPfeezlx4gTPe97zuPPOO7n11lvr5mCYN202m8Xlci2rAdnMm9h0wRVFWTX6n83rmqDpXJFcXlfYisXiIKVuOIxmxLBynELfwMwyXcLlctLb14diNSep8RWDYVq5X7fLwd7uFly/8zu4PvOZss+0rVvJnj1bRlrzer20t7ezsLCA3+9n165dq1+oy4Smabz//e/H7XbzsY997OlmKKBhLDYW+Xye73//+9x77708+uij3HrrrbzmNa/hOc95Tk1PQVVVTp06RTAYZMeOHZhtBcwakMXFRVpaWiza+UYaDlOkp7m52Tp2PUhm8kzH0xSK+hO8VCqyEI1ZzYTC4TCRlhYC/iZgeQpVr2jVGBwcxOdroqen2yLDCSmrZ1ps0dPK4rO+tiAhvwflJz/Bc+ediGxW/7ypieK73kXpgx8s25UpJVAqlfD5fGuSA1gPNE3jAx/4APl8nr/5m7/ZNEMxNjbG2972Nqanp1EUhf/+3/87v/3bv71Ru28Yi81CsVjkP/7jP7jvvvv46U9/yvHjx7nzzjt5wQteYKVSC4UCJ06coKenp6ZQjqZpRKNRZmZmSCaThMNhOjo6Lps1WCqVOHHiBJ2dnfT29q75+1JKoqkc88ksRZuQcElVicfjRKNR8rkcwVCIlkhEVxcHo3WAxsWL/YRCQbq2dJXtt1oVKtReznjcTvZ0LWWLHF//Oq4PfAByOUpvehOlP/iDpQI39OtpN5DVuCcdHR1rKltf7Tp96EMfYmFhgc985jObyvidmppiamqKG2+8kcXFRY4dO8a//du/cfDgwY3YfcNYXAmUSiV+8pOf8JWvfIUf/ehHHD16lGc961n8x3/8Bx//+MdpbW1dfScssQZnZmbK9DxbW1vXZDhM/sa2bdsuuyemqmnMJbMsLGYr5PkEJVUlmdR7l6TTaZqbmwmHgkxNTdPa1kZnRV1NmfaFnQBG7R/Z9CrqQaWhqISp0G7SuespW18JUko+/OEPMzIywj/+4z9esdIAE3fccQfvfve7eelLX7oRu2sYiysNVVX57Gc/yx/+4R+yY8cOtm/fzh133MHLXvYymprql6k39TxnZvT2fIFAwKKdr3RTZjIZTp48yd69e9es5rUSCiWV2USGeCq3LCUKuuBNPB5n+NIlhKIQDAaIRFqXUrI1oHsUFTszUOlVrARN0zh16hShUIjt27fXtb29bL1aVe9KkFLyF3/xF5w9e5b/9//+36aVAtTC8PAwz3/+8zl9+vRGCeZcs8biqiplbSYUReHs2bM8/vjj9PT08Nhjj/GVr3yFj370o+zcuZM77riDl7/85av2F7G32bNXnV66dMmqOm1vby+7Sc21+qFDhzZcccntdNDb2kxrs4+ZeJpUtrw7vaqWmJycZPuOHUQiYdJG+8KJ8XE8Xi+tkTAhW8m6CatQrCKFCtAZqs+4moYiHA6zbdu2ur5TWbaeTCaZm5tjaGjICpC2t7dXZedKKfnEJz7BiRMn+NKXvnTFDUUqleK1r30tf/VXf/Vzoaz1jPUsakHTNE6cOMFXvvIVvv3tb9PT08Mdd9zBK17xiroKuEyYdQpm1anb7aazsxOXy2XJ/q3Fg1kvFrMFpmIpCkWVfKHAxYsX2Na3lWAV8lE2lyG6ECO2QkrWhKLo+p0+t5PddXgVmqZZtTVbt27dkHNLp9MWndvUS21vb6epqQkpJZ/+9Kf54Q9/yL333rtuqv96USwWedWrXsVtt93Ge9/73o3c9TXrWfzcGQs7pJScOXOGe++9l2984xu0trZy55138spXvrLu+IaJdFpvNjQ3N0cwGGTLli3r6lmxHkgpmZiL8dBjJ+jbuq2mt2QGL4WAbC5HPKb3LpFSIxJpKUvJgh7O6G0LEmpa+RxMA9zW1kZfX9+GnpsJM0A6NzfHJz/5SZLJJPPz8/zgBz/YtOxKLUgpufvuu2lpaeGv/uqvNnr3DWNxrcOsgrz33nv5+te/TiAQ4NWvfjW33347HR0dq0bsJyYmmJqa4vrrr6dUKpUpiJuFbpejdL0SUqkUp06d4sCBg+RxMl8RBIUVaRWU1BLRaIxoNFqWkm0JNbOne+V4i6qqnDx5clMNRSU++9nP8uUvf5m+vj7OnTvHd7/73csOIK8FP/nJT3je855X1sfkT/7kT3jFK16xEbtvGIunE8xS+fvuu48HHngAl8vF7bffzh133EFXV9eyPif2JjiVQbl8Pm8ZDk3TLK7BRi1RkskkZ86c4ciRI1Z9S1HVmImniady1nY1O5VV1JGYKdlYLErQqdHXpRs6szDLDlVVOXHiBB0dHetKC68HX/rSl/jCF77Agw8+iN/vp1Ao4HK5rlqB2Cbgmj2RhrFYBVJKxsbGuO+++/jqV7+KpmmWJkd3dzff+9732Lp1a11CwoVCwaKdF4tFy3Cst4gtHo9z/vz5mvGRXKHEdDxNKldY9derpI/7PC62tzdbVbLJZJJQKERnZyeRSAQppcUf6enpWdf414r777+fz372szz44IPranz9NEHDWDwTIKVkamqK+++/n/vuu4/h4WGOHDnC//k//2fNNQ+mzsXMzAy5XI62tjY6OzvrJiktLCzQ39/P0aNHV13eLGYLTMfT5AsVmqBV1iYCgRSSbe0hmn1LQcNKxapCoUBnZyd79uy5IryGr3/963ziE5/gwQcfXFMgei14xzvewYMPPkhHR4fVxPsqoGEsnkmQUloFbR0dHdx///1Eo1Fe8YpXcMcdd6xZks/UYpiZmSGTydDS0kJnZ2fNepXZ2VmGh4c5evRo3VkAKSWxdJ7ZeJqSoQlaSxejyeNi55Zw1f2oqsqTTz5peRebpVxmx7e//W3+7M/+jG984xsbylmpxI9//GMCgQBve9vbGsaiCq64sXj/+9/P17/+ddxuN7t27eJzn/vcpj0pNhMDAwPs3r3beh2NRnnggQe47777mJqa4rbbbuPOO+9csyZHZb2K2SYhEtHFZqamphgfH193xaymSeaSRmOkGtWl2zrKvQoTpVKJp556ip6eHrq6lujjpnDx3Nyc1Uahvb19QwK6P/jBD/ijP/ojvvnNb9LW1nbZ+1sNw8PDvOpVr2oYiyq44sbiu9/9Li9+8YtxOp3cc889APVqEz5tkEgk+PrXv24tVV7ykpdw5513cv3116/JcJj1KmazI7Pd44033njZvIKiqjEbTxOzBUGhtldhGore3t4VpQez2ayV4rzcgO6Pf/xjPvCBD/CNb3zjimU7GsaiNq7qMuSrX/0q9957L//8z/+80bu+ZrC4uGhpcly4cIEXv/jF3Hnnndx0001rMhzDw8PMzMwQDAYtWnRnZyctLS2XFTOwgqAGE3R7Z4iAt9wQlUolnnzySbZu3bqmSWsGdGdnZykUCmsqHvvpT3/KPffcw4MPPliz+G8z0DAWtXFVjcXtt9/OG9/4Rt761rdu9K6vSWSzWb797W9z7733cvLkSZ7//Odz5513csstt6zYn+TSpUtWHxNFUax6FVM3wu/309nZuWq9ykpYzBZYzBbobinPzBSLRZ566qk1G4pKmMVjs7OzVs/TWnIADz/8MO9973v52te+dsW4GyYaxqI2NsVYvOQlL2F6enrZ+3/8x3/MHXfcYf392GOPcf/99z+TcuR1I5/P873vfY97772Xxx57jGc/+9nceeedZZocZgOeYrHIwYMHq14nsw2gSTuv1SVtPTANxbZt2za0I5ymaVZcxkzJtrW1EQwGOXv2LO9617t44IEH6ipE22g0jEVtXBXP4p/+6Z/41Kc+xQ9+8IMrUj9xraNQKFiaHA899BDHjx/n1a9+Nd/85jd5y1vewvHjx+s2qGa9ytzcHG632wo2rjXGUSwWefLJJ9m+ffumto40u9CdPHmS3/iN3yCfz/P7v//7vP3tb7f0Uq8U3vSmN/HDH/6Q+fl5Ojs7+dCHPsSv/MqvXNEx0DAWS/j2t7/Ne9/7Xn70ox/R3t6++hd+zlAqlfjhD3/Iu971LtxuN9dddx2vec1rePGLX7zm7ILZJmF2dtbKUnR0dKxar1IoFHjqqafYsWPHFfuNzpw5wzve8Q4++MEP8uSTT6KqKh/+8IevyLGvMTSMhYndu3eTz+etQq1bbrmFT33qUxux62cMPvOZzxCLxXjf+97HQw89xL333su///u/c/DgQe68805e+tKXrtkjs7dJAGpK3F0NQ3H+/Hn+23/7b3zxi1/k0KFDV+SY1zAaxmKz8ZWvfIUPfvCDnDt3jkceeYSbbrrpag9p3ZBG20I7NE3j0Ucf5Stf+Qrf+9732L17N3feeScve9nL1kx9tterqKpqGQ6Xy8WTTz7Jrl27rginAXS+ylvf+la+8IUvcP3111+RY17jaBiLzYapnP3Od76TP//zP39aG4vVoGkaTz31lKXJ0dfXZ2lyrLWJjpnenJ6eJh6P09nZyfbt2/H7/ZseeB4eHuZNb3oT//AP/8CxY8c29Vjf/va3+e3f/m1UVeVXf/VX+V//639t6vEuAw1jcaXwwhe+8BlvLOyQUnL69GlLk6O9vZ077rhjTZoc+Xyep556ip07d1rl9WabhM7OTpqbmzfccIyNjfHGN76RT3/60/V0Fr8sqKrK3r17+d73vkdvby/Hjx/ni1/84kYJ7Fb1BC8D16yxeMbK6v28QAjBkSNHOHLkCB/84Ae5cOEC9957L6973esIBoOWJkd7e3vVGzr//7d3ryFRf2kAx79ndO2C3S/TtFNCSSZFVPwD3Y3KLS/8M0XDmvGfJRVSsZB0pdUXRSzVi/SNxZLblS6GWEjZhSwqCC0jpbVMws0yaywtK/MSjWdfpIO2XWZsnN+Mcz4wL5zR+T1HhmfOOb9zntPWRmlpabdaoQaDAavVSn19PU+fPrUV1dXr9d/cqu6oFy9eYDKZyMrK6vVEAXDnzh0CAwOZMGECACaTifz8/B4ni5cvX3L9+nXMZjPt7e22xXWdywWcfbi2u/CoZGHP+g1vJoRg8uTJpKenk5aWRlVVFXl5eSQmJuLn50dMTAyxsbGMGTMGIQStra2UlZV981BoHx8f9Ho9er3eti6itraWiooK236VnlTjtlgsLF26lIyMDGbPnu3M5n9XbW1tt8VdRqOR27dvO/w+nT2InJwc6uvrAWztP3r0KLdu3QJg06ZNLjsm0ZU8KlkUFhZqHYLHEEIQGBjI1q1b2bJlC8+ePSMvL4/k5GSklISFhXHlyhVOnz7909PjO0+nHzVqlK0ad11dHZWVlQwZMoTRo0czfPjwnyaO169fk5CQwK5duwgLC3Nmc3/o20V/HO8ddf6NwWCgqKjI9vyDBw94+PAhGRkZnD592uXrQ1zF4852UxwnhCAgIIANGzZw8+ZNMjMzOXLkCAMGDLB9y1dVVf34GMQOndW4g4ODCQkJwWAw0NDQwO3btykvL7fdYflaQ0MDCQkJ7Ny5k4iIiN5o5ncZjUZqampsPz9//tzh/SbNzc1kZ2fz/v17xo4dS1NTk+21ESNGkJ6ejr+/P/369ePVq1ecOHGC2tpap7XBHfSZZHH27FmMRiNFRUUsXLiQyMjIX37PS5cuERQURGBgILt373ZClNoTQlBUVERubi7Xrl0jPz+fkSNHsnHjRubNm8eePXuorKy0K3EIIRg2bBhBQUGEhIQwbtw43r17R0lJCffv38disdDS0kJjYyMJCQmkpaU5q06lQ2bNmsXjx4958uQJnz59Iicnh5iYGIfe4/r169y4cYO5c+dSUFDAmzdv+PDhA/BljqJz3cubN29ITU2ltbXVZRXEXKXP3Q1xlt6eQXdHDQ0NtpocFouFyMhI4uLi7CoZ2FXnMQkWi4Xk5GQaGxuJjo5m586dmtUuuXDhAqmpqVitVlauXElaWlqP3qewsJCPHz8SFxfHqlWrGDx4MOnp6eh0OoYMGcKxY8eoqqpix44dQI/ulLjt3RCVLL6jqKiI7du3c/nyZQDb0uNt27ZpGZbLNDY2dqvJER4eTlxcHNOmTbM7cTQ1NZGQkMCiRYtoa2vj5s2bXLx40RNPNu9216O9vZ2wsDDMZjOPHj2ivLycMWPGsG/fPtra2mx7abr+jQPcNll41ASnKzlrBt1TDR06lKSkJJKSkvjw4QMFBQVkZmZSWVnJ/PnziY2N/WFNjubmZkwmEytXrmTFihUAPf42dwdd26nT6ZgxYwYBAQGsWbOGsrIy/P39uy2Ik1J6ZFL8kb7VGidy1gx6XzBo0CBMJhO5ubkUFxcze/ZsDhw4QGhoKFu2bOHWrVvdJjVbWlpITEzEbDbbEkVf0fm5+Pz5MyUlJQBMnz69W4lF6JufFZUsvsMZM+h90cCBA1m8eDEnT56kpKSEyMhIjh8/TmhoKKmpqRQWFrJs2TJiY2NZvXq1y+LKzc1lypQp6HQ67t692+vXmzVrVp/rOfyMd7XWAc6YQe/r+vfvz6JFizh69Cj37t0jPj6e/fv3o9frWbdunUu/XadOncqZM2eYM2dOr16ns039+vVz2WY7d6HmLL7D19eXrKwsIiMjbTPoavv09/n5+REVFUVUVJQm1w8ODnbp9Uwmk0uv5w5UsviB33//vdfWBbjJgTaKYjc1DNFIcnIyly5d0joMj7JgwQKmTp36f4/8/HytQ/MKqmehkTlz5lBdXa11GB5F7Q3SlupZKIpiF5UslD6hN/YGKd2p5d4acoMzKhT347aruVTPQlEUu6hk0UVLSwsXL178ZjUuZzObzYSGhlJZWYnRaOTgwYO9fk1F+RVqGNLBarXi4+PD2rVrGT9+PNu2baO1tZW6ujoCAgK0Ds8hNTU1LF++HIvFgk6nIyUlhfXr12sdlsM2b97MuXPn8PPzY+LEiRw+fFizLe4upIYh7q7zQOG3b99iNBqBL9vRU1JSOHPmjJahOczX15e9e/dSUVFBcXEx+/bt4+HDh1qH5bDw8HDKy8u5f/8+kyZN8tYTytyGShZdSClZsmQJV69eZdq0aVitVk6dOkV8fLzWoTnEYDAwc+ZM4MuO0eDgYI8s8RYREYGv75elQCEhITx//lzjiLyb1y/K6lrJqL6+nvPnz3P58mWys7OJjo62bUnuYSETzVVXV1NaWuqSkvu96dChQyxdulTrMLyaI3MWfZoQIgpIAcYDxUCqlPKztlH9GiGEP3AD+KeU0i3HUkKIQuBbB22kSSnzO34nDfgNiJfqA6sZr+9ZCCGCgHjgD2Ar8B/g30B/oEkIMRQwAzXABSllu0ahOkQI8ScgDzjhrokCQEq54EevCyFWANHAfJUotOV5/Wrn+w34M18+jAWAAfCXUjYJIXRSykbgEbABD0mu4su46iBQIaXM0Dqenuro7W0FYqSUzVrH4+28PllIKU9IKf8upazreOo1UC2E+EuXXkQLUCal/CSE8IT/2V+BJOBvQoiyjofra/D/uixgEHClow3/0jogb+b1cxZCCB8ppfWr58YD7zt6FQgh/gGUSCmvdPQ2PGIooijO5Anfkr3q60TR8dwzKWWjEEIvhMgEkoH/drymEoXilTxiDK4VKWWdECIbsADTgSptI1IU7Xj9MERRFPt4/TDkZ4QQOtEXD4FQFAepnoWiKHZRPQtFUeyikoWiKHZRyUJRFLuoZKEoil1UslAUxS4qWSiKYheVLBRFscv/ADAnXr79KbQXAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "visualize_fun(linear_module.weight.t(), 'Dataset with learned $w$ (PyTorch GD)')" ] }, { "cell_type": "markdown", "id": "composed-election", "metadata": { "id": "NIHkSu_Q4sqX" }, "source": [ "## Linear regression using SGD \n", "In the previous examples, we computed the average gradient over the entire dataset (Gradient Descent). We can implement Stochastic Gradient Descent with a simple modification." ] }, { "cell_type": "code", "execution_count": 12, "id": "retained-demographic", "metadata": { "id": "FmnDVz6N4sqX" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "iter,\tloss,\tw\n", "0,\t0.01,\t[0.02993712 0.40586257]\n", "20,\t7.11,\t[-0.21203727 0.6746018 ]\n", "40,\t0.13,\t[-0.5276561 1.248206 ]\n", "60,\t0.00,\t[-0.64190584 1.4578575 ]\n", "80,\t0.01,\t[-0.7105578 1.644668 ]\n", "100,\t0.01,\t[-0.84861267 1.8285881 ]\n", "120,\t0.00,\t[-0.884992 1.8483113]\n", "140,\t0.00,\t[-0.8915096 1.8892854]\n", "160,\t0.01,\t[-0.9286745 1.9184113]\n", "180,\t0.00,\t[-0.93960834 1.9385221 ]\n", "\n", "true w\t\t [-1. 2.]\n", "estimated w\t [-0.94389266 1.9486643 ]\n" ] } ], "source": [ "step_size = 0.01\n", "\n", "linear_module = nn.Linear(d, 1)\n", "loss_func = nn.MSELoss()\n", "optim = torch.optim.SGD(linear_module.parameters(), lr=step_size)\n", "print('iter,\\tloss,\\tw')\n", "for i in range(200):\n", " rand_idx = np.random.choice(n) # take a random point from the dataset\n", " x = X[rand_idx] \n", " y_hat = linear_module(x)\n", " loss = loss_func(y_hat, y[rand_idx]) # only compute the loss on the single point\n", " optim.zero_grad()\n", " loss.backward()\n", " optim.step()\n", " \n", " if i % 20 == 0:\n", " print('{},\\t{:.2f},\\t{}'.format(i, loss.item(), linear_module.weight.view(2).detach().numpy()))\n", "\n", "print('\\ntrue w\\t\\t', true_w.view(2).numpy())\n", "print('estimated w\\t', linear_module.weight.view(2).detach().numpy())" ] }, { "cell_type": "code", "execution_count": 13, "id": "recreational-alarm", "metadata": { "id": "KaVDLxVS4sqY" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQ8AAAEDCAYAAAAr7YFFAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAACXTUlEQVR4nOy9d5gkZ3X2/Xuq83TuyZtzlLRKKzACEYwQFhJaDDbBBr9gbOMEfCa9BhuDDRhHwAls4wCvwYCEAkIyApMVQBKSNu/OzE7OqXPuquf7o8JU9/Tkmd0V9H1dgp3u6qrq6npOnXCf+wgpJQ000EADK4VyqU+ggQYaeHaiYTwaaKCBVaFhPBpooIFVoWE8GmiggVWhYTwaaKCBVaFhPBpooIFVoWE8GmiggVWhYTwaaKCBVaFhPOpACHFaCPGiRd7vF0K8dAX7W9H2lwpCiP8UQnxkA/b750KId673fjcKl+PvJYR4XAhx+FKfhx3LMh7GxcwLIdJCiIQQ4lEhxNuEECv5/Ib/GOt1HCnlYSnl99Zznz+rEEK0Am8C/tn427yXMkKICSHEfwghAsvYT8b2n2bbR0YI8Ssb/T0WOa/nG+shKYSYFUI8IoQ4WrPN64QQPxZCZIUQk8a/f0cIIYz3l7O+/hr404v53ZbCSjyP26WUQWA78HHgfcC/bchZNbAkhBDOS30Oy8T/AR6UUuZtr90upQwA1wJHgT9aaidSyoD5HzBo7sP47wvLPZn1vG5CiBDwdeDvgRiwGfgwULRt8y7gU8BfAR1AO/A24EbAbdvdUuvra8CLhRCd63X+a4aUcsn/gH7gpTWv3QBowBXG3/8XuACkgTPAq4zX/5+xXR7IAO9dbHvjvfcBI8Z754GfN17fBHwVmAL6gLfbPlP3OLb33wzcb/u7B/iK7e8h4Gr7913k3PuBdwMngCTwZcC7nOu3xHdY8JrY9vM+47hFwLnYuSx2LOP9a4CnjON9GfgS8JEFvkMG2Gr8+22ABNqNv98DfHaBz30H+NWF7iX0RfV1236+WvP5vwc+uYz78SDwPSABnAZeucR12wrcbVybGeAfarZf8vcFrgcSi/zuYSALvHqt68t47VvAry1nzV6M/1ZtPIzXB4HfNv79S8bNqgCvNS5a5yIXp+72wH70hbzJ2G4HsNvY7ifAB9Et9i6gF7hlqfM03ttl3FiKcZwBYMT2XhxQ6iz2eufeDzxunH8MOAu8banrt9R3WOwa2vbzjHHj+xY7l2Ucy21cg/8PcAGvAcosbDyGgEOAAE4C3cAB4+8e4MgCn5sCjtb7jYzvcRr4M+PvTuM7R4y/ncAkcN1i96Nx/j3A+43v9RJ0g7i/3nUDHMBx4BOAH/ACz1/p7wuE0A3P54BfAKI1778cqADOta4v4++/A/72UhsN87+1JkxHjYuLlPJOKeWolFKTUn4Z/ea6YaEPLrK9CniAQ0IIl5SyX0p5Ad29bZVS/qmUsiSl7AX+FXjdck7U2D4NXA28EHgIGBFCHDD+/qGUUlvBd/874/xngfuN/S6FRb/DMq/h30kph2R1GFDvXJa6Xs9FX3SflFKWpZR3AU8scu4JIAC8DH2hngIi6AtkREp5fIHPRdCvux33CiESwMPA94GPGd9/DPgBuhHF2Pe0lPIni5yX+V0CwMeN7/od9HDi9bZt7NftBnTD8B4pZVZKWZBSPlyzzyV/XyllCng+uhf2r8CUEOJrQoh2Y5MW4/wr5meMfEbCyHHctMT3staXgTT69bwssNb4bzMwCyCEeBPwB+ieAug/ZstCH1xoeyllj5GZ/xBwWAjxkLHddmCTcdOZcAA/XMH5fh94EbDH+HcC3XD8nPH3SjBu+3cO/WZcCot+h2Vew6FlnstS12sT+qK3azIMLHLuceN83gn8BXoYGAV+B/2JuNjngjWvHZNS/u8C238O+G30xfir6KHjUtgEDNUY/wH0+9OE/bptBQbsi7oOlvX7SinPoud1MB5E/wV8Et1wzQAtQgineSwp5fOMbYdZOudorS8DQfR79rLAqj0PI6O8GXhYCLEd/cf+PaBZShlBfzIJY3NZ89lFt5dSflFK+Xz0BSDRb9YhoE9KGbH9F5RS3mrb9VLiJKbxeIHx7++jG48XsrDxWE/BkwW/wzKu4UrPZ6nrNQZsNjP+BrYtsr8EeozfKfVKVAo4AlwB3LvI504A+5Z5zhj7ukoIcQVwG7CcZOgosLWmOrENPW9mwn7dhoBt6510llKeA/4T/ZoAPIaeY7ljpfuyry/bywfRw63LAis2HkKIkBDiNvTk2n9JKU+ix40SPb5FCPFm5i4gwAR6zG1iwe2FEPuFEC8RQniAAnqyUkWPQVNCiPcJIXxCCIcQ4oqasljtcWrxfeDF6PmCYfSn8MuBZuDpBT6z1D5XgsW+w1LXcD2PBfqNXQHeLoRwCiF+kUXCTHQP4v9jzstIAe8A/llKqS7yuQfRjfOyIKUsAHcBXwQel1IOLuNjP0bPlbxXCOEyODq3o9+j9fA4uvH8uBDCL4TwCiFuXO45mhBCHBBCvEsIscX4eyu6x/Ej47sk0Ksv/ySEeI0QIiCEUIQQV6P/3vX2WW99YayH69CTppcFVmI87hdCpNGt9geAv0V3XZFSngH+Bv2GnACuBB6xffbPgT8yYr13L7G9B71UNY3uOrYB7zdu0NvRY88+4/3Pome06x6n9gtIKbvQqwY/NP5OoScRH1lkASy6z5Vgse+wjGu4bscy3i8Bv4jucsfRE7R3L7LLOHqY+0Xj7xR6/P2vS5zK54FbhRC+FZz+59C//3JCFvO7vBI9aTkN/BPwJsMTqLe9eW32oCclh9G//0qRBp4D/FgIkUU3GqeAd9mO9Zfooeh70ZO/E+icl/cBj9r2teD6MvBK4HtSytFVnOeGQFSHvA00sP4QQnwMmJRSfnKZ228DzgEdhoH/mYcQ4sfAr0spT13qczHRMB4NXFYw8hZ/C4SklG+51OfTwMJ4trAUG/gZgBDCj+7WD6Dnohq4jNHwPBpooIFVodFV20ADDawKDePRQAMNrApryXk04p0GGth41JIELxs0PI8GGmhgVWgYjwYaaGBVaBiPBhpoYFVoGI8GGmhgVWgYjwYaaGBVaBiPBhpoYFVoGI8GGmhgVWgYjwYaaGBVaBiPBhpoYFVoGI8GGmhgVWgYjwYaaGBVaBiPBhpoYFVoGI8GGmhgVWgYjwYaaGBVaBiPyxANdbcGng1oaJheRpBSoqoquVwOIQQulwun04nD4aB6NlMDDVx6NIzHZQIpJeVyGVVVkVKiaRqqOjdKxuFwNIxJA5cV1iKA3PCt1wmaplEqlZBSIoSgXC5b/wasqeSapjE+Pk4oFCIUCjWMyc8GLtsftuF5XEJIKalUKlQqFYQQKEr9FJQQwno/n8/j9/vRNI18Pm8ZjYZn0sDFRsN4XCJIKSmVSmiaZhmHlUBRFMvYmF6J3Zg4nU5cLhcOh6NhTBrYEDSMxyWAqqpVoclKFrYQYl41xu6ZwFzitVKpWNuYxsTpdKIoSsOYNLBmNIzHRcRyw5S1otYg1RoTIQROp9P6r2FMGlgNGsbjIkHTNMrl8qrDFBP1PI/lfKbWmFQqFcrlsvW+oig4nU7cbnfDmDSwLDSMxwbDfOr39/ezefPmJb2NQqHA+fPn8Xq9RKNRQqHQunso9YzJ8PAwUko2bdrU8EwaWBYaxmMDYeduDA0NsXXr1kW3n56e5vz58+zYsQNVVRkfH6erqwu32000GiUajVr7XU+YxkQIgcPhsM7b7pnYE7ANY9IANIzHhqGWu7HUtt3d3aRSKa6//norNOno6AB0byQejzM8PMzMzAzxeJx8Pk80GsXv96/7QjaNiIl6xsRMvjqdzjWFYQ08e9EwHuuMlSZF8/k8J06coKWlxTIcpVKpahuv10tnZyednZ309vbidrsRQtDf3082m8Xv91ueic/nuyjGpFQqUSwWAb1s7HK5LM+kYUx+NtAwHuuIlXI3JiYm6Onp4dChQ1ZIshSEEHg8HlpbW9m8eTNSSrLZLPF4nAsXLpDL5QgGg0SjUSKRCD6fbz2+2rxzMI2JGUKVSiXL6JnJV3uY08BPHxrGY52wEu6GpmmcP3+efD7P0aNHcbvdVe8vVlGpfU8IQSAQIBAIsHXrVqSUZDIZ4vE4XV1dFItFy5hEo1E8Hs/6fGHb8YFFjUkikaC1tRW3290wJj9FaBiPNWKlYUo2m+XkyZN0dHRw4MCBDQkxgsEgwWCQbdu2oWka6XSaeDzOmTNnqFQqhEIh3Zj4fLhUFfz+ZeVmlnt8qDYmFy5cIBwOWzmThmfy04GG8VgDVsrdGBsbo6+vj8OHDxMOh1d1zJXyPBRFIRwOEw6HrSpOKpUic/IkU9/9LlJV8YbDFJ73PDxbtqzqnJY6X7NaA3NNfrVhjr0vp2FMnh1oGI9VQEpJsVgkHo8TiUSWNByqqpLP55mYmOCGG26wFtJqsBqSmB0Oh4Oo201bTw/akSNoTie5yUkK3/seAy94AZOTk0QiEaLRKOFwuCpRuh6oxzExr2dtAtZezWng8kPDeKwQZpiSy+Xo7+/n2muvXXT7TCbDyZMncTgcHDly5LJYCCKbBSnB40EBAp2dqLOz7OzspGXbNhKJBNPT01y4cEE3Nka+5GIR1mqNSaNj+PJEw3isAPYwxSRTLQQpJSMjIwwODnLllVdy8uTJdcsprJUkJv1+UBQoFsHjgUwGzedDejy4XC5aW1tpbW0F9MRnPB6vS1gLBAIXxZhomkahULBeaxiTywMN47EMmBTzcrlsJUUVRUHTtLrbVyoVTp8+jaIoVWHKeiUl1wy/n8qLX4zze99Dqip4vWRf8ALdoNTA7XbT3t5Oe3s7UE1Yy2QyFo1+IwlrixmT8fFxOjs78Xg8DWNykdEwHktgIe7GQsYjmUxy+vRpduzYwaZNm6zXTY9hrTf2engeAHLHDsqvfz3k8+D3o05MLEsN205Yk1KSz+dJJBIMDAyQyWTw+/1WzqSpqWnDjcnY2Bjt7e0NYaRLgIbxWAS1FHP7TagoStUillIyODjI6OgoR44cwe/3V+1rvRb9usLj0f9jdV6REIKmpiaamprYtGkTUkpyuRzxeJze3l5yuRyBQIByuUw+n98QwpqUsspA1BNGahiTjUHDeNTBcrgbQgjL8yiXy5w6dQqPx8MNN9xQt0KxXOMhpWRsbAyAWCy2IgLZpYYQAr/fj9/vZ8uWLRZh7cSJExtKWLMbg3rCSPVU1sz/GsZk9WgYjxosl2Juvm6Sr3bv3m01stXDYjkSE6YRMvtETp06haqqVhgQiURW/b0uBUzCmtvt5siRI/MIa+VymXA4bBkTl8u1IefQUFnbGDSMhw21AjmL3URmObGrq4trrrmGpqamRfe9lMdg5kp27dpFa2srlUqFnTt3oqoqiUSCeDxOf38/5XIZn8+H3+/fkNLpRqKWsKZpGslk0krA1hrKtfBhFsJyVNbsYU7DmCyMhvFgLkw5ffo0HR0dSzapFYtFTp48iZSSo0ePLmsBL2Q8pJQMDQ0xMjJi5UrsT0WHw0FzczPNzc0ADA0NkclkmJiYoKurC4/HU1U6fTbd6IqiVOmU1BpKwHp/IcLaWkO4xYxJLpejUCjQ3t7eEEaqg59542Hnbph/L4aZmRnOnTvHvn376O7uXvaTv57xMA2Ww+FYMFdSC6fTaTXBgV46nZ2dZXBw0Kp2RKNRYrHYhiQoNxK1hrJcLpNMJpmZmbEIa3b260Z4XXZjUiwWSafTNDc3zxNGahiTn2HjYeduAEtyN6SU9PT0kEgkuO666/B6vXR3dy/7eLXVmXQ6zcmTJ+eVdFcKr9fLpk2brGqH2Z7f3d1NoVAgGAwSi8WIRqPzkq+13+9yg8vloqWlhZaWFkAnrCUSCSYnJ+nu7sblclEqlUgmkwSDwXU3JqqqVo24gPrCSD+rKms/k8bDLg+4HO5GoVDgxIkTxGIxS7BnpbBXZ4aHhxkaGuKqq64iEAgseI71jrNUu769Pd9MUM7OzjIyMrJkTuFyv+ndbjdtbW20tbUBumfwk5/8hNHRUdLp9LqHcCaT2I6VqKz9tBuTnznjsRh3w+FwzDMeU1NTdHV1cfDgQWKx2KqPK4SgUqlU5Uo2IiFohz1BWS/5qigKkUiEWCy2YZ7HRno0brcbl8vFwYMHAV2VLR6PWyFcU1OTZUxWQ1jTNG1Jb2a5Kms/jZKNPzPGYzncDbvnoWkaXV1dZLPZuoI9K0WlUuHUqVPs2LGDzZs3r/oGWgvPo15OIR6PMzExwfT0tBUGPFuSr7Xemc/nw+fzLUpYs8s1LgVVVVdcPl7MmJj33SOPPMJVV13F5s2bV7Tvyw0/E8ZjudwN03jkcjlOnjxJW1sb+/fvX7Jku9QiGx0dJR6Pc/DgwTXlN2B9SWIul8sKA0ztU4fDsa7J143s51ls3wsR1laisFYvbFkp6kk2fulLXyIWizWMx+WOlcgDKopiub2HDx9ekpS1VL+KqqqcO3eOcrlMe3v7ZV/9cLlcdHR0rDn5asdGGo/lhBUmllJYsxPWIpEIbrd7Rftf7jmAria3UK7r2YSfWuMhpSSVSlEqlZaViVdVlbGxMYrFIs95znOW5a7WVlDsyOVynDhxgk2bNrF161bOnTu3Lh7DxaKnrzX5amKjPY/VLu7lENbM/a83Yc0UqX6246fSeJjcjZmZGYrFIqFQaNHtTcEe041dbpxrVlBqXdvx8XEuXLjAFVdcYckNrmTRX4pcw1KLfKnkqxDCCnHszNeN9jzWa9/1CGunTp0il8vxzDPPAFQZy7WEM+a4jGc7fqqMRy13o171pBYjIyMMDAxwxRVXUCgUSCaTyz5eredhV0W/4YYbqozQenkMl0tj3GLJVzvzNRAIbNj5rndYYYfD4cDtdrNlyxaCwSCVSoVEIsHs7Cy9vb1rIqw1wpbLDPW4G4sZj0qlwpkzZwAswZ5isbiksbHDzt0whze1tbXVVUVfyaKfmpoin8/T3NyM1+td9vlcStiTrzDHfB0ZGSGdTnPq1Kl1Z75utLiSqqqWh+F0OpckrJmey1Jhsqqqa67eXQ74qTAeC3E3HA4HqqrO295kd27btq2qbLqczlc7zO1NLoh9eFO+VGZ0NkN7xE/A6140P2L/HmZ5OBQKce7cOUqlEuFw2EpUXi6ex1Iwma/Nzc2cPXuWHTt2rDn5WouN9DyW2n89wlo8Ht8wwtrliGe18agnD2hHrTGwN6HVY3cuZ4HXore3l0KhMI8LMhbPkitW6JtI4ve6KJRVAosYpmKxyPHjx2lububIkSNUKhW2b99uJfJmZ2cZGBhAVVWcTieJROJZ0VVrJh3XI/lab98buShXYpw8Hg8dHR2WLMNChLVkMrniXI0QYj/wZdtLu4APSik/advmRcB9QJ/x0t1Syj9d9kFWgWet8VgOd8PueZTLZU6fPo3L5VqwCW0lnoep5dnR0cF1111XdfxUvki2ULb+zhbKjMRz5FWFcKwZn7s6ITs7O8vZs2c5cOAAzc3NVd5SbSJvcnKS8fFxS5DY1BCNxWJrkv3bqIVYb7/LTb4ulU/YaM/DHrasFPUIa9PT03zgAx9gcHCQX/mVX+HlL385b3zjG5fcl5TyPHA1gBDCAYwA99TZ9IdSyttWdcKrwLPSeCxGMbfDNAamVsbOnTvp7OxccL+KotQNc2oxPT3N+fPnCYVC89iiUkrG49l5nxFCkCmW6BlLEGpy0x7243E5GBgYYGJiwmq2WwpOp5Ompib27NljaYiaLMp8Pm9VjOqpkF0KLMcoLZR8NfMJbrfbCnHsIcClDFtWAjth7c477+Tmm2/mfe97H+fPn1/N7n4euCClHFjzia0RzyrjsdLRjkIIMpkMZ8+e5eqrr15SsGep6ow5OjEej3P99dfT09MzL8yJZwqUKyogAZvmqRCYu07lSsTTOSaGB2iP+JetCWJ+J/OYwqYhag69TqVSzM7OcurUKTRNW7fy4mqxGo9moeRrLfP1YuQR1vsYZpn2qquu4qqrrlrNLl4H/PcC7/2cEOI4MAq8W0p5erXnuRw8a4yHpmlMTEzgcDgIhUJL/qilUsmazfq85z1vWYtzsbClWCxy4sQJIpGI1Vlbu72mSSaSOTTDnghhawwTClLq2+ZyOXp6uuns7MQZaWV0NkNbxI/buXYqtD0cqC0vOp1OYrEYsVjsoiXx1iMcWkh2YHx8nGKxSLlcvqy8rcWQyWRWXaYVQriBVwJ/WOftp4DtUsqMEOJW4F5g72rPczm47I2HPSmaSCTweDxLznk1cwi7du1icHBw2U/1hYyHub99+/ZZw5Bgfvl1KpWjopqfF+hvCRzC/C56yDM6OsKePXvw+/1IqXsryadOEBvspbU5hHj+jbAAA3El1Zba8mKxWJz3BDfDgY3CeleG7MxXj8dDNpslFosRj8frar5udOfySrFGjscvAE9JKSdq35BSpmz/flAI8U9CiBYp5fRqD7YULq8rW4PaMMXpdC4ZVvT29jIzM8O1116L1+ulr69vwe1rUa8609fXx9TUVN2chH37sqoyncrVOytUqf//zMw0TqeLw4cP43A4sZySE8fh299hNhgkfr5I9OlTBH7nN/DE1ndRezyeqpkr2WyW2dlZurq6SKfTljcSjUbXddFtJMPU4XDMG+S9muTrxcIa2aWvZ4GQRQjRAUxIKaUQ4gZAAWZWe6Dl4LI1HvUm0DscDos9WovasGI1N4ndGJRKJU6ePEkgEFgwJ2Ev7U4mcmiyOs8B+sIpFouMjY3j9XrZv28/QhEImAtvnnwSWlvB40EAsyMjTDx1hubrr6Y56MPpUKr2t15MVfMJvm3bNnp7ewFIpVIMDAwsSDdfKS52Y9xqk68XC6bHt1IIIZqAm4Hfsr32NgAp5WeA1wC/LYSoAHngdXKDCUGXnfFYjLvhcDiqZpaaMKsf+/fvt1z01cBcmIlEgtOnT7N3714rabfQ9pqmUShVmM3o56WIOaMAkkQ8Sf9AP83NMf1pLsxwRiIQ6JGNrVoDYOxjKpVnNlMgFvDSEvLh2EBVKtOYmGGZuejMkrDH47HyJSspCV+qlnwT9ZKvdv6FmXw1NT7M/W3Uultt2CKlzAHNNa99xvbvfwD+Yc0nuAJcVsZjKe5GbTVE0zR6enpIpVJcf/316zJEqFgscu7cuWWNUzA9lbF4xnhFVBmO0ZFh4okkBw8eJJ1Ok8/bDYwwPgEcPQrf+l+UgB+tVIJQCLllCwJQNclMOk88U6A55MOlyYvCMK1ddPl83kq8ml2hpjFZSht1Iz2PlYZXteMyc7kcs7Oz9PT0WMzXjQxxTFGinwZcNsZjOdwNOw/D7CVpaWmZR9JaDcyBS1JKbrjhhmWPU8gUShQd1aSvSqVCT08PXq+Hw4cPAQpCZPTvhjQMjPGEA+SVV6F4vcjeXmSTH/XIVWDkV8ztNQkTiRzFQp5StoimSRTl4rncPp+PzZs3WyVhkyFqT1LGYrF5JeFL7XksBjv/ws58jcfjjIyMkMlk6O7uXtfk609LRy1cBsZjJdwNkzFqxq/2XpK1IJVKcerUKXbt2kU+n19RdWZiNkOkee5myOWy9PT0sGXzFmLNppcpUcxSrRBGbFINuXcf2t59CAFC08AmqGt3NFRNYzpTons8TmvQRzTgvehxuxCCUChEKBRix44dVCoVi0JvloTNfMl6ts3XYr1JYnbma0dHB+fPn6e5uZnZ2dl1S75mMhkrH/NsxyU1HsuVBzQhhGB2dpZisbgiXdGFnlD1Bi6ZicPlIF2oUCirespCSiYmpxgfH2Pf3r14ffaQx8htIBGArCGQ6a+Z56R31SYTCcLhCMFQEKdzfmt/uaIyGs8ylc7TGmoi6vdcsuYrp9NZlaQ0S8LDw8MkEgkURcHn8637LJmNNkwOh8MKzWB9kq/ZbJbt27dvyDlfbFwy41GpVBgcHKSzs3NZhiOXy3H69GmEEFxzzTXLvmkWkgpczcAlOzRNMp0poGkaqqrR399HpVLh8OHDOJ0OatMSiqJ7EPMJZMKyHpo2t5+Wlhay6QyTU5NoqkowFCYcDqEo5mBm/fuUKxqjsxmmUznawn4i/vUZHr0W2EvCU1NTTE9Po6qqpR1qdglHIpE1zaddi5LYUjBnttix2uSrHY2cxxpgD1MGBgaWJQI7NjZGX18fe/futcqIy4UZ6thvBLMlf/v27asWoZ1O59CkoFwucfr0aVpaWujo6EAxyrAIqgyITk83X5gjkJlGpFQq0dXVTSwWo6OjnUqlQjAYooNNqGqFTDrN7Mws6XTaCN0mCIXCeDy6x1GuqAzPpJhKOWkLNxFuWpkR2ajchBACj8fDtm3bLO1QM8QZHBwEWHUosJG9LcsRP15u8tWeVF5NzkMI0Q+kARWoSCmvr3lfAJ8CbgVywP+RUj61ooOsAhfVeNRyN5aCqqqcPXuWSqXC0aNHEUKsKKyA+cQvUzlsNQOXTJQrKlPJHLlclsnJKfbv329JHUqJFaIoQhjcD9ANRr1kB6RTafr6etm+YwfhcMS4PnObOBxOwpEo4UiEYqHA4OAQoM+tLZVKBAIBIuEQgWCIIipD02mm3Xnawk0EfZeWrl17LWu7hO0KZN3d3SsqCV8u4sqwePL19OnTVCoVjh8/zvj4+GrP+cWLsEV/AZ2Kvhd4DvBp4/83FBfFeNTKA9aO76t3MU1d0S1btrBlyxaLU7ESsR6Y8zxUVeXMmTNWNWWhzPlCYY4dE8ksA4NDJOIJWlpaLMNRzfGYK9sqZmhSkykVSEbHxpmZmeHAwYN4PG7bOQhqT0EIgUSgOBTa29tpb2tDk5JsJksimWB0bAwQhMMhQqEwuWKZJo+T9rCfwCUyIktdy7WUhDfS86gXtqwEtQLLqqqSTCb50pe+xDvf+U6CwSBf/OIXrZnDa8QdwOcNUtiPhBARIUSnlHJsPXa+EC6a52FWU2qfQrXuoZSSkZERBgcHufLKK6tUplcj1qMoCplMhuPHj1cZosW2X+ymTGdzPPbkM/h8PrZs3UomnTIKKNLiblRDoBmEMKMwCwhUVaW39wKKonDo0CHju5mfkPMMB4CwGSZzW4eiEAoFCRjXqVKpkEolmZqaYrC/H7fXy2A4TGdrM1vbo/g9q88xrAYr9Q5WUhK+GAnT9YLD4eClL30p//zP/8xnP/tZmpqalhTmtkEC3xRCSOCfpZT/UvP+ZmDI9vew8dqz33iY1PJar8H0CswfyUxiKoqyqHewEhSLRc6fP8+RI0eW9WMt1lmbTCb530d/QlvHJmKxZpLJBJrUQxVFCMs02KHnNHRvQdWbXCgXi5zv7qK1tZX29o7q4wuoaMY+Dfslpb4fTdbS04UVJulejzQ6Z5uJxZqRUlIoFEilUpzp6ubEmQqtsTB7tnSwqb31orToryW0qC0Jm30rpmdSLBZxOp1IKQkGg+tqSDbKqzEZpkvNBKrBjVLKUSFEG/AtIcQ5KeUPbO/X++IbziS8pKVap9Npkb5MrsVap8ab0DSNc+fOUSgUOHjw4LKt/ELGY3h4mK7efrbu3GOVG4XQtxU2r8OkfkkECGklRs2cRzKZpK+vn927dxEOhWr6Yaq9F9NGKDXJVzuE8Z5JPDMTsIoQaAhL0aq9vR1NU8lms5zqG+OZsz20hnx0trVs+Kza9VrUtX0rx48fx+v1Mjw8TDqdpqmpyQpx1loSXouK2GLI5XIrTphKKUeN/58UQtwD3ADYjccwYI9/tqBremwoLprxqHcDmY1uk5OTjI6OWlyLtcIcuGSWgVfyBKk1HvZcyeZdB6ho0lrw+neSVcQvQ6YHhMnpmPve+XyO4eFhrjh8GKfLNZcTMbwGPcFqbj1nVEyPRhHVGiGmN2KH6QXNPXjsYaLDmJqmG9JyucR0Ns9sop9MKkEymaRcLhOLxdaF6q+fz8YlNYUQtLW1sW3btiqdD3tJ2EzOrrQkvBrq+0bsVwjhBxQpZdr498uAWm3SrwG/J4T4EnqiNLnR+Q64xJ6HEIKzZ88SDAZXxLVY7IacmJigp6fHGrjU3d29LGlBE3bjkcvlOH78OJs3byYYa2V4Zq6HRWCEKpqs6xUIKawFb5alVVXlqquuqjl3YXkNssrTnPNGMLwRiyOCSTtbeFFq0l4y1retNk7gcrnB5cYhImQrEI6GqkSUzDJjOBxe9VN4I/tw7KGFvUvYrHaYE+CGhvR0gJkvWU5JeCPCFilX1ZfUDtxj3DNO4ItSym/UdNQ+iF6m7UEv1b553U56EVwy4xGPx5mammLr1q3s3bt8waOFqiHm2IJcLlc1cGm14xRMFuEVV1xBMBiia2y2ajtpbCvRqJUc1E9Ufzmby9Nt5DcUoSc4rQWsaZDPgduDcLuQhpFwKFAxz6dmwesVF72vRZNzx5l7f87IGM27VvJVq3vjSlQgU1SZyFTYvrmNw5u34BD6bzQ9Pc2FCxdwuVzEYjGam5tXLLR8KXpbFioJm7+rWRKORqP4/f55+9mosAVWdj2klL3AkTqv2ztqJfC763JyK8AlIYmZAjudnZ1LqoLVoh7pyz5wqXaq/XKmxtkhhGBgYKCKAj+VzFGuzN+HopjELzGvFV9KnUo/NDTInj17cLncJBOJuQ8nEyhf/jJiYhIcAu0Vt8MR/R7R5Jx3Mb+CoxtPTUrD2NSGO/Nhln0F80Md+2clEM8USWSLugxANGZJHJg6on19fdZcGTO/sFhIsJEs0JV4BwuVhPv7+8lms1VzZDwez4aLK/804KLmPIrFojUT9ujRowwMDFCpVJb+sA2m8TBv2HoDl+xYriI66AJA09PTxGIxq1O3omoLKoTpuqT6yrMWvAKaCkPDQ6TTaQ4dOozL5aJcLhuL3vge99wL09PI9naUYhHlvvtQO9rBqL7oIc/85CkYzoZlVERV7kTK+V6QnQovbZ6Inmidv72UktmMLgMQC/hoCfnm6YiaQsvDw8NIKatCnOXweNYDa1ngC5WEzZBNSonL5VpTyFaLUql02WusrgQXzXjE43FOnDhRpQO60ES3xWB+xq7lsViT3HKNhykAFA6H2bRpk3XDTyZzVLS5BKU9fyAENsq5vuArpQo9Pd34mpo4ePCgtR89zDAf8RIGB6G9HUVqaAbFXExPI9s7jNhYQ5NzN625/hZoyrWOL9ENmLQ5SrWJ26oqDtVdu/rrukGSwHQ6z2wmT3PQVyVIVCu0bA8JvF6v5ZVsJBcD1ickqlcSPnHiBJlMhqeeeqqqQW4tJeFsNrukRsyzCRfNeHi93nk6oE6nk1KptKL9OBwO8vk8p0+fprm5eUktD4fDsegxzM7a0dFRrrnmGkZGRqwwp1hWmc3kbdvq/y+ERNX0Uq20rdJcLkd3TzdbNm+mubnF2NbgadgXqRDIlmZIpxChMEJVEWoFGQyiaSoXLvSSyaRxOl2EwyHC4TA+n55nUMwd1mDuZWEZDsXIjywU0swxYOf6fmvLxeZ2U6k8s9kCzQEfzUEvDtsT3+l00traSmtrK1JKKyQwjbvf78fhcKy7NupGwRxyvWPHDvx+vzVKcq0l4bUop1+OuGi/ZFNT0zz90dV4HuVymTNnznD48OFl6SIsljBVVdUipR09ehSHw1G1/XgiU2edCit34HDMNbvNzEwzOjLK3j17q54ueulUIpXq3hbt2DGc//VfaJMTCFVDu/H5lNo76D5zllgsxvZt26ioFZLJFOPj4+RyeZqamggGg6iaZngNpsGYq8jYYZZ/hZDzwhNRKiJVDbxeNMPTUAwGcL28qiIEqiqZTOaYTedpDvloDvjmCRIJMTdLZsuWLZYAtV0b1Vx4yxmhcalgD4nsoyTtDXBmSdjM/yxVEv5p6qiFS1yqdTgcy855SKkPXEqlUtZYxuVgobAlm81y4sQJtm7dypYtW6q21zSNbKFEOlekXv5gbiEKpKYyONhPNpvn4KGDVdobxpnPPcmlrTKzaTPq7/0eTE+jeb2kvF76zp1l586dhIJBypUKLsVljU7Qn+g5Zmfjuud15iyhUIhIJKyPcKjz3efOFaykrqrifOghHI88AhLUK6+k8upftK6xbuzshsn8FnN/VDTJRCLHTLpAa9BHLLi4IJFdG7VUKlkDoc+dO0dTUxPNzc3EYrFlTcy7WFiInl6vAc7M/wwNDS2a/2mELeuI5XoeZqLVzEesJElWr9pickGuvPLKecxT03iMJ7J6E5oAzfZkt+cPKhWVXD5PJBrj0MEDevPeAvkDmMtVmKQw2eSHbX4mJycY7x/gwIEDeL1eNIM7oihYKutC0Z/oXq+XdDrN3r17SadTTE5O6jelr4lgOEgkHNH5G3WgSXAcP47re99D7dwEisBx4jgyFoWDhxCKnU9iJDqNDGs9b0RVNcYSWaYz+WWrmrndbr2pr73deorPzMxw7tw5yuWypfURjUYvyYQ7E8ttjFMUhUgkYtHNzflC9pKwKTeQSqVW7HkIIbYCnwc6AA34Fynlp2q2eREXecg1XGKGqZ2evhDi8ThnzpyxEq29vb0rCnXsxkPTNLq7u8lkMlVcEDsURSGZLaJ5dYblHDFL9xjMv7PZDD09PbhcLrZu3WI9qeeIWYBNr3Tu+xuJUykAjd6+fiqVMldccRhFcVRlQ6VBNBM6YRXN5gXpUn8xYs0xNFVSKOQN6nsf5XLFypXoCle20Q1DQ2i+JoSxMGQwhNLXBwcPIassn2Ew7GSRGgNi2pZyRWM0nmU6nac13ESkaU7VbLFqi/0pvm3bNqvz1CwJX4oJdyZWW8lxuVxW/gewZgk/+OCDfOxjHyMUCvH5z3+eW265hfb29uXssgK8S0r5lBAiCPxECPEtKeWZmu1WPeRaCHEQnWi2W0qpCf2G+QbwX1LKzy/0uUvueSwUtkgp6e/vZ3JykmuvvdZKTK00T2KGLeZcl1gsxrXXXrvoDT2ZytHitXskc3xORcDExCRj4+Ps27ef7u6u+d6FaUQkSGOFiR//mC3334/ygx/AS3+e4u69dHWdJxKOsHv3TouLsQCPCw19sSpCMY+ic1I1/Zx9viZ8viY6OjpRKxXSWV08aLinB29Fxb9pE+GWFhzNzVAq6mGIECjZDJV9+1AEdQmrFhmthj5fjxpfqmiMzGSYThpGxO9ZUam2VvavdsJdIBCw3l8v+vxCWC9+itlb9PrXvx5FUXj66acZGxvjxz/+Ma985SuXcx5jGN2xBkX9LHrHbK3xWDWklGeFEOeA29Cp7h8Dzi9mOOAyMB71DEGpVOLUqVM0NTXNG7i0GuNRKBR48sknlzXXJZUvUyzPN2iKgIqq0d/fT7lsegpK1fAmO6SYK++KJx5H3HcviqpBKo367/9J70tewqbnPIdoNGozGPXLIqZR0TtobfulvkSBy+UkEo4S6x/Ec889lMtlSi43Q7fcQtYfYEdLC/7hEVxuN1pzC5WbXwrJVB3bYa+8VPNJFqrgKAKKFZXhmTTTqRyZQpnoKj2G2gl3mUymiothEtfWk4uxkcjn8+zbt493vOMdq/q8EGIHcA3w4zpvr3XI9SeA/08I4QJuBF6y1Acuu7DF5Fvs2bOnrlu32NS4WkgpGRsbI5VKceONNy5ZVlM1jdlcCaRmdJrM9ZcUi2W6uruIRqPs3LnTaoqTxvu1j20FoRPXJYinn0EEQ2j5HEWHQj6fZ1ehgMtGapub5WJqeej7rHrCSwmFot5PAwipzQ2OmvvWaAhEfBbXXXeiBoMobg++dIr9D/+Q/LvfQ3r7NsZ7esgmU1Ta24nk85QqFb3igh5R6YlTpS6l3fzO8yozopp0ViirjCVySFcaty+wJkEiIYTR1Bdk+/btVCoVnnjiCaanp+np6bHEiGOxWF26+eWATCaz3FBlHoQQAeCrwDulbS6tgTUPuZZSflMI8TfAnwMvlFIuucguqedhr4RIKRkYGGB8fHzRgUvL9TwqlQqnTp3C4XAQCoWWVY+fTOq6pKom55KlUpLNZOi5cIEdhkygCXsew/40nueNeL1o5TIVTSWfzxMJBiAUxDYVt+oJr8930fepvwticgLX5z6PmJhgJ+D4/d9D3b4DMPgcUjPOWfcQxGxc/6Dp3gdDyKlJHIUCoWiU0NEbACiViiSTSTKZNOl0itmgXsEJhUIIx2K3x5zhMK+Tbsjms1VLFY3+qRRNHl1fNeBdO8tSURRcLhf79u0D5nILJt3cXj69XFidqy3VGt7AV4EvSCnvrn1frt+Q60eBp5fbkXtRjUe1kM3c4jMHLnk8niUHLi2HMZrJZDhx4gQ7duygra2Nn/zkJ0ueW7GsMpPO2/pVjEFL4xNMTU1y4MABPB57KbH6iWzS0y0xdNvb5Re+kMLTT+MrFIhEIshwBHn9nIZtPW6F3dlQpIrr7/8e5dw5cDgJFAs4//EfUT/4JxAIWN6B2TQHIMNhvfGuXAGXE5nLgK8JzefVmSqGd+H1eHC3tlEoFAiHwwihkEqlmBgfB6FYkoZmM1y9kEUzSHDCCquE7b25v3PFCv2TKfxeF23hpjWpmtUmNM3cwkrp8xcTqxk1aYgb/xtwVkr5twtss15Drg8B/7HcjS853U9VVZ544gl27txJZ2fnktsv5XmYSuumhKGpn7oUJhJZoyqpGFqpKn19fUhNcuiwXgmp8i4WWPDCKKSYeYpcLkdPMsnW3/5tUj96nNDePcjDh8EoEVsRyjxIg24uYGIKx+nT4POhud1QqaBc6MVx8iTKT36CMj2Ntns36i++Cs0f0D2RlmYqd9yB8/779XNxOin/2v8BxWFL6prl2LnvFPS4iczMQKVMces2korCxMQE+VwOX5OPcDhMKBSeV6nSy9RmqDUngmQqrNmRLZTpLyYJeN20hZvwuVd+Gy5WDVkJfb7eiISNkhFY5bS4G4E3AieFEM8Yr70f2AZWd+16Dbk+DJxa7saXzHhIKRkeHiafz3PjjTcu+6IuZDw0TeP8+fMUCgWOHj1q3dzLiX1zxTLpfAmBnmGvVMqcPn2G1pZm2js6rFKnyX8wZ7As/N30/4/HZxkaHGTP3r0E/H6GCgXkVUeqNzb6ZRTb4gOMCXPGNqWibkgcDhRNQzoUKBRx3nUXNPmQ/gDKyZOQyaC97W1WOCSf+xyK+/cjMhmdy9Hkrzm0MGj2xokUCrj/+V9wjAyjCQWn14vr936X5l27kFJSyOdIJpNcuHABKTWCwblysJ1JI6XJwpWLVlvS+RLpfImQTzci3hUYkZVUcRajz5selxniOJ3ODevHWY3nIaV8mLp1sKpt1jzk2uCTJKSUmSU3NnBJwpZKpcKZM2dQFAW/378i1l0941EoFDh+/DhtbW0cOHBgxT/8WDxjLdxcLksiEefAgUNEwqE6VQX9SW3evHYDb3ompohzKpXk4KHDuBdaFMJsq6+uZKhS2kSTQcaa0TZvRkxMIIWCKJYgGtbJW6YWZmurztfI5wwjoe/TEYkgo5F5T9Mq3Q/DKHqffhplaBCts1P/nvE4zq/dT/ltb0MIaGry42vy09G5CU1TSSZTzMzMMDQ0iNvtMZ72Idxuj+5tCF0saR6FHaqMbypfIpUvEW7SjYjHtfRtuVoeRi193j5HxqTPh8NhTOGe9TQilzM9XUo5BOxayWcuuudRO3DpRz/60YqUqmuNx+zsLGfPnl0RZd2OVK5IrlixKjPT01MEg2HDcMyvosCcXqhZkTAzDZoUqGrFEM9xc+DAQesGn5v3ZBLE5ioyc9AXfLFYZHRkFH/ATygUxhkIUPn1t+L6whfQikUKpRK85jU4v/kQiqZXV6Ra0YlfNexSFTCZZvVyElVIpxFG45pEgteLSCX1c61hzyqKg2g0QnMsiqrpQsvJZJL+/n59YFUgSDgSRjO69KrIc4K6oVraMiIe2kJNuF0L3xPrtbBrRYNKpRKTk5OUSiUef/zxdaXPNxrj1oCRkZGqfATMV1BfCiZ93E4iq+3WXS6klLrXoan09FzA6XSwd+8+BgYG5hRAhRkDm+FETcJQCFRNw2GUYLu6uuno6LBEZ+YRvwyyJrL+qIZ0KsmFC710dHZSyOeZGJ9AKAqhtlZif/AH+IpFeicn2H/0BpSJcZSnn0ExyrulY68CWy6iqjVfzlHLzOqI3YhIKVF37UI+8CDEE+D1ojgUyjccxSz/zoepDyLwN/nw+bx0dHToQsvpDPGErotaKpUtinaT1zOvImPsyjqnRLZIMlck6vfSEvLhds6/NzZKrMcs+cbjca644gqy2Syzs7PrQp83RYd+WnBRjYfX6503UsHpdFKpVJZdTjNZqc888wxer3ceiWwlmMnkSWeynO/qpqOjnba2dkqlElLT5trbrcSffnObXocJxeiWTST0p+6e3XsIBANz3a5Ub2+VN6emEH19egL00GFwu5mcnGBycpKDhw4ihK6bsWkzVCplUqkko4kkuVyesstFPBEn9MuvxX3kakglkZ2daDt32gSBTMNRj/olLWWxKk3T3j5AImZmEaqKtnULlRe92Mi/aPMWvd0waoZ3o+gXhWA4TDAcplyu0NraQrFYZGhoiHKpiD8QJBIJEwwGdUo+pjNSfZ1mMwUSuYJhRJpwOeZ+54sxs8Wui7oe9Pl8Pt9ojFstWlpa5uUrVsoYzWazpNNpduzYsazqjIlaN1fVNM73jdDX38+ePXsI+AOGLqmo28Kvt6yD1Ztiu9HHx8aIJ5IcPHgIt9tt9pLVEdrRDY2jvw/xT5+GShmHlIjOTmZQCPX30rFlC9rrX0+pvR3HN76B48xZXJEIrmN30Lx7N0jJiZOnyOXyjI+Po7ichHbu1DU/JJhBwkLjGvQFb+RSMJmqAlQN9w9+gNy1G7lX504wNYkYGkQ7cHDus+YH615k/fimAppm5A186QzRoSE6vR7KBw+RKRVJJJKMjIzgdDgJhQKEw1G88yofEk0TzKQLxDMFmoM+moM+nA5l3ejj9bCQV7NW+ryU8lnBhF0uLnmpdiXGY3R0lP7+fnw+34oMhzlpzt6s9eSJswyPTc0teHTj4HA46rIqhcUABZP5Wamo5PMFXC6XNfXNhPn0t4ZBSfM8NJS7vop0uaA5htQk5YcfIRQO4t6/Hy2TxfHpT+PauxfHyZPISARHfx/KP/4jxXe/B8JhnIrCtqlJxMAg5WiEmVCYsbFx8vk8fr+fSDhMKBzS9UmqqjgSWRMqmZR3gUSRWrWjIhQUTdPzJpYhlJbIUO2+jM3mSrYIfMND+O++G6FqCKmh7NyJeOc7CW6dG/+QSqUYGxsjn8/R5A9Y3BK3y2l5RpYgUaZALODFscZxkIthuR219ejzMzMzddXnVzPtEEAI8XL0IdYO4LNSyo/XvC+4BEOu4RJUW+adwDI6a80BTqVSiRtuuIHHH398Rcc1iWV6GbbCU88cZyqnVSU0waZ9Ied7HrUU9GKxSFdXFx6Xk23bts+72cwEo2ZbcJbxymTA46GiqqTicWLFIkpkO5rDhQg6EZOTOB77EQQCKD09evLC34RjoB/1qiM0P/IwrlOnEQ4HTrVC56HDtPzmb8DAAOqPf0xeU+ndtRs1EtErIJEwfp8PMS9Bq0MIvQxcOHoDgSeeQAsEoFiEcAh1x87arQ0aew113np3LvyQQPM3HgJFQUQiev2otxfHT55Cfe5zAT3H0NzcQnOzrlmSy2ZJppJMTEyAEIRDIcLhiEVSUzXJTDpPKpVCqZTR6lRz1orVjJq00+d37NhBpVIhkUgwPT1Nd3c3H//4x60ZQIcOHVpWyGWsi38EbkYf7PSEEOJrNR21l2TINTwLPA9TGb29vb1KE3Q1xzA7a72RNra3hudtZ05Zk9bfc+GKPbmZSqXo6+tl185djI2PYS6XuXVUmwwVVglXSg151ZWo3/kuKY+HkNeLcDqRRsLX8lJKRcSFcXC79VBgZgZmZqBQIPbkTxBbt6A5nIDEee4s8vvfx3HvfQgpaULSfOoUmXe8k6TDwdjoGLl8jlAwSCQcJhAI4nA6QVXBtkjyr3wl7o4OHOfPQyxK5dZboU6MbiWNDcMoFL0kO1eCnfvujkwG6fcjhcH9AMhmbVfGlhkSAn8ggD/gZ/PmzVTKZZKpOc0Sn89nUecrqkY2V6ZrdJaWUBOxgHfdjMh6JGOdTqcl5ATwF3/xF7zhDW/gwx/+MFJK7rzzziX3YTwke4zxCxhDne6guqP2Di7BkGu4TIzHQm3509PTnD9/fkFl9OVCURQmJycZHh5mz/6DTGQqmDbI7knWp13ba4tCF+4Zn7CEe8YnJvTqD1j7FMwXBdK/q4KmSUaP3oAyPEz78AiK24V8y1vgkYd1HoeUyKuuRJ45izI7iygWcZTLqE4njI/rCx7QrEUv0BQHju9/H+FxQzAAKDA5gff4M7hedgstLS0IJKl0hmQyyfSZM2z52tfwzc6iNLeg/uZv6EbE4UC9+WbUm2+eu3a2vhnjillVG/P4ZmJWGNvakdm3l+CZ00h3G7JURioO2L3LqDgt3J0rAYfTSUuzfe5unlQySW9vH6VSEafTRTyZpKyqqFKjPbz2aYOwMTNbtm3bRiQS4Stf+cqyPzMyMgLzB1jXehWXZMg1XAZhSz3PQ0pJb28vs7OzXH/99Qsmn5bjhUipjyEslUocPXqUodms8br+vim5V1t5sJ+z7jVo9PcPUCqVuOKKwzgcDqtMaVU3qkKU2oSl/kd//wCKorDr99+OVBRLelR77nMRw8Pg96MdOoj4h39ATE/B2DhSKCiqqiuA3foKctu34Z+YhHAIsjmIRg3WaQEQCCl1HZGS3hhp8lCCwSBBfxOef/83ZL5AKRajkoyj/sVfkn3Lm3G53Ph8Pr3SYJzxXN+MeZ3qs2uFua1RcTHzHlMvfjEtwSDOp58Gr4/KW96ih0LSGOStzc+/znXzzhlhh6Gk5vM10d7RydTUFJlMmtlZfSJcpSNCpbVlXebUbkQZOJPJrGZGbd2Xa/6utwhWnlxZBS6551GroF4ulzl58iR+v5/rrrtuway3qqpLKnGXy2VOnDiBEIL9+/eTr2hkC9WdxtXexXxoUlIuV+ju7iIcDrNjx06cDoFZkLE30oEhFGwzIqaEYUWtkMlkaW9vY+vWbXP5D4wwZesWtC2bMZlU5Ze+FMe3vw1O3SOQTifS4cD15BOM3PoKYj09KL0XkDt3UTl2B8rpM7i/9KVqwtiRI4BE2ObLiGQKpqcRzc34pETz+RAzMwQzGQqFPF1d53EoCqFwmHA4gtfrtfp4hM3nWLBsa1RcrOY7t5vSm34N9c1vqd4e5q6hMBO3+hWpZ8RNuQAzz6IzRf20t7fTHPAS9irMzMxYosSRSITm5mYikciKvYiNMh4rJYgZ2rpLDbC+JEOu4TIwHvawJZVKcerUKXbv3r2o7sFyjIfJZN29ezczMzNIKRmPZ+tvbFfIwh6DQyaTpaenh23btluhk2a00AopqhdmVQJxbp+5bIbu7h58Ph+trW1zHlOhAGdOI8pltL17EYEAKAqaoiD37kXdfwBlfAy8XmRLCyKdhmIR6Xaj/vIv6SMpzWrE82+kJASOxx5DetyoL/8FtK1bLUKa6TfIpiYQCrJURnM7QdUJd0o4QltbG4FAwBjNmGBkZIRisUggENDnmoSDOBT9mlc3CdYZuI1+XKnJquFVc1dn7jpXD62a30hnbq/N7Riz+V8R0BpuwulQaGpqYuvWrRYfY2Zmht7eXlwul8USXc6ozI0Ycr0aavrRo0cB9gohdgIjwOuAN9RsdkmGXMNlYjxUVWV4eJihoSGOHDmypHu33M7aqw4cIDgzQ25sjJmmECXFM48ZXRteSHQGuSIkE1PTjI2Osm/ffpsrbMT8JsVcYa46U+eun52NMzw0xN59exkZGZlzRXM5XH/7N2gjowhNg5lpCAaRHg+8/Ba4/ZVoN78U5Z579Pb6UhmpSYr79iOLRYskNde1KlBvvBH1ec/TT8SmW1rlCXg8VF73Wtz/77/QhABFUPn5l1JuaUFqGo4HHsD7yCME3G46jr2KylVXkslkSCQSjI2N4nA4rY5Vr9eDwypBzl+QZkkXMccn0aSsa2xgri9IMVwRu3dT/bvprF7FIWgxeB921PIxCoWCZUjy+TyhUIjm5uYF58hsRM5jNcrpxrn9HvAQeqn236WUp8VlMOQaLoOch5nMDIfDHD16dFkWf7HO2q6uLvL5PDfs3Yv3j/8YBgfpyGbpuvrnUH7nd8HlqqFnz7/xJZL+/kFyuRxXXnGFJRYM9QZPK6hGpcF+h0spGR0dIZlIcvDQQUPRXF8UigD52KPIoWHo6ICBfkgmkUJB+ppwfu7zMDSC9vrX6VWMH/8IQm7U217BbCSCMjWFanoMioLTIRAoKP/zIM5vPARSor7g+ai/+ItIhwMr4JAgKhUc57ugWECUK6jXXI36ilfgOHsG/1fvwvX443pjXD6P+9P/hHzPewju3kM4pDcJ6uJBKZtX4icSjhAKmzNY5q6lnS9j8UkWeegLBdBqRYYW8Gyk3l8TCy6d3/B6vdZoSXNUwszMDAMDev7JHOBtskQvl7AFQEr5ILqBsL92yYdcwyX2PHK5HGfPnsXpdHLllVcuuwxbb5BTqVTi+PHjxGIx9u/fj+NTn4KBAdi8mYl0Ac6dQ3nsMbSbbqpyt2uJTpVKhWKxCAL22zp0F6Kn6+Qp/VzM1vqKqnHhwgUcDgeHDh2yyjDmyElNgiOd0vMZSEimwOVG5HIovT1IVeL43vcQI8NU3vd/ES/Tqx+Dw8Nk43EOHDhgXAMVTdWoqBLlicfxfu1+RCyK6nDg+O53EeEw5ZfdUn3tvvsdHI89iuzoBCTK+S6c99zNtvvvxzs0BAgc0zOoV16BJgTKM8eRu/dY18zt9tja2zXDK0kyMjqiK7qHIwRCQZp8TQbDto5xNvIXVrXXfF2zb2s30mYp3GaYNI3mgHee17EUakcllEqlKpZoMBikVCrNG8mxVqymHf9yxyUzHuaA6t27dzM2NrYi/kat55FMJjl16lTVHFzR3w+hEGVgxuHB4SzopU4bap9wuVyOnp5u3G4Xmzdvrjonc6LaXGLP+Kw5wkA/KoVika7zXbS2ttLR0V7teluVGYl24CCOh74J+TyiWIB0Wn/0+rzgUPQcx2wc5fhxyjfeSH+fHrvv37/fenorihPhBFWTOLu7wO1GdboAieL1oZ08hbz5ZuyjF5SubqTPp1dnAMXjQTz4PzAxoZdrJbrYUFc32ubNSL/fCu1qmwIVRSEYDBEM6gvNlDQcHhrRe1iCQVRVRUoNIarDANP0m8nSagbvHOwGvlrqUdK8DK9jKbjd7qppcOl0mvPnz9PX18fAwIDlleh9OKv3RhrGY40wF09PTw/JZNJMCDE8PLyi/diNh5krqdU9lQcPonR1MRGIgiwhSmXk9m3W+/bwQ5MYE78G2btnL/0D/WiapDrstc9gAbNLVQiBNLygTCbDhQs9c1qnQlaVLUFAMonj5Ak0oPyqV+H83OeRxRKK04koFNA0FVpbdeMxPY1aLHD+/DlisRgdHR16idOgnJvrWFEExJoRlQqKQ9HzBeWSruOhaaiGZ+RwKMj2NuSpk9a30kpFRKWC0CTS50Pk9Nm8Ip+HSATt555reQ/WMCiDFFbrUeheSRvtbW2omkY6nSFudKQ6nU4ikQihULiqA9qsds0FkQs/ROzHD/mcK/Y6loI58NqcI+PxeKzpdubMXdOYrHT0w2pKtZc7Lqrx0DSNp556ilAoZA2oVlV1xfNqTQX106dPU6lUuOGGG+YluLQ3vpFi/yCz5/pwlsrkr7sG/w03GO/OGQK7cM+hQ4dxu1w4jJAA5trbq2az2DwWRdGf/NPT04yOjrB/v04e04scc8lMRYAzPkvgc5+HfF4vO4ZCyOZmZGeHziLt70dMTUEgAKkUqsvJebeHzZs3Ew6HMXZlDI0CpwIVQwlMe8mLUZ56CjExgUCihcLIV70Kh9OFomloUtOv9c0vw3fmLMr4uP7BzVuotLfh+vKXkYqC4vMiCwW0llaKH/gASiRSwzcQxqwY8+/5RkTPUyiEQiHcHjeHDx+mWNS9kqGhIUqlEsFggHA4QjAYwOlwzBkGYRVU6nBljGuJIOx1bXhvi8vloq2tjba2NosvZB/9YPauRCKRJc8lm82yefPmDTnfS4WLajwURWH//v1V7ttyBI1roWl6TmHr1q1s3769fsjT1MTwO96NOjFNIpEg73LiN8hboC/sWuEeh2IsduMOFlUudZ3zMJ7Is7MzSAmHDx/GsYDiuCYh/MMf4sikjXwDiJkZyGWRfj/OoSE0VUVfnJLS9m1cuPJKth+9Hl+dLL0EVGn4MwK0QIDK+96LON+F0DS0vXsgGDRa5B04cIDU0FxhSu97H/T2AqDu2I7icFI4eYrgubN6GToQoPL7v4cIh+s2CeonUG0YzWtRTyBZSl1o2dPWRltrG5rUSKfTJJNJRkaGcTidRMIRo4LjxeTILCRc1Bz0kczXT8CvB+r1ttRrz4/H49boB4/HU1UOroXZtPjThIue8zBFiU2s9AaIx+MMDw/T0dHBjh07FtwunS+RKVYgEgFNQ8tmwUrgQalY4Nz583R0dBrCPXNhgF5B0SzDIWryHCakpjE1NYXT6eTgwUM4FHPafH1j48hlqbjduJD6+Ei3C9p3ofz4x8hKRe9xCQYoKwp9V13FjpteiMfjtp7KdtjLoNI0Ir4m5NVH6tMLJYCCooDS1ARXHNYNhaZSLpcZPHYHhz6fQpSK4HTh+OJ/I3fuRG7ZMu+71H4/Uz2dup6CcXjbNVSEQjikCykLoFAskkwmqrySaDiCPxDA6XTolH3DiDgUQXPQS3x8Y8SAYHkkMYfDUdW7ksvlqnRRI5FIlWjQWlTEhBB/BdwOlIALwJullIk62/UDaXQBuYqU8vrabdYTl5znsVxIKRkcHGRsbIydO3cuuf14Yk7HVbHCEH3RxRMJ+vv72b17D8FgwKqSWGQlo3XehDnq0c49KBaLnD9/Dr/fj8/XZNHYhcEorXXnNU3Fcd31VE6fJS/1qW7eXA75S69B6enWE51OJzmfDyWVYncgiHC7rRBFUezDn7AMR9U1MhIxQhE1oYaRm8G2A/QciFopceHCBfafOqX32bS16mcej8OD36D8ljejKKKqpdz0OKqPPffKnAyA+Xe1sbGMtPEvj8dDW5suxqRpql7BSSYZGhnG5XIRDoWJRCK4PR6agz4jrNw447Hclnw76umizszM0NfXx7e+9S3Onj3Ltddeu+y2ihp8C/hDKWVFCPEXwB8C71tg2xevYl7LqnDRjUetaPByoKoqp0+fRlEUjh49ytTUFNnsAmxRYDaTp1CqWAZhjkIuGR4ZY3Z2tlq4p2Yh2Lc3yWBmt6giIJlK0dfby85duykUCpTLc/R6Iap5IFJqmGK67p9/KQ5Vw/vQQ1TUCmMvfCGjwRAHWlvxZLMUPF6cArxeD+UOG8O2qsogrFGWtRBgTHuziSebZU4539ik02l6ey+wa9dufABOp65gBgiHgigVEAJUVbN4JU6nw2iDr3HrmRu3IKW0hYdywfDDHNdgz3MoisMa7wAYuZIEA4MDqOUyzp0dOCotG6oktlahoVpd1La2Np588kn+67/+i7/5m7/h61//Olu3bl1iL1Xn803bnz9CH7VwyXHZeB4LWeR8Ps/x48fZvHmzdcEXY5hqmmQikcPOE3A6HKiVChd6LoBgnnCPuTiFIU4sDNm9erNZxicmmBgf58DBg7jdbkqlYlVvi52eLqXUvRXDs5ECtJffgvZynXvRCrQBibf+BurffQpXIgFCMHPbK/B0dOCpHywZ+Zi5hadfv7nvom9l+geGF1RzaROJBMNDQ+zbtx+v14t64/PxnDiBlsvp51zR4AUvwOVyoWn6d9BU1cjLQEXTR10qDkUvBVsWoPp8BFgGuAq2PIm9obA2RLJ7JS1BDy6txMzMDDMzMxSLRdra2mhubl5zM9y801tHw7R9+3b8fj9//dd/zY4dO9bqMb0F+PIC70ngm0IICfyzlPJf1nKgpXBZGA+T9FWbpJqZmeHcuXMcPnzYIvXA4sZjOp2jotYQyMoVEvE4W7dvo62trYr3QNUT0yRzKdaCMVedOQ6zVCpy+PAVKA6d4OWoCons4wwkmqYa/Ai9TGsPHHQI0uk0A6rKzo9+jKZikaLLSV7TGBsYMMhKusuu8wxElXEQYs6TcyiibnJzLo8zt0gnJ6eYnJpk/4GDuIwxB/Lo9RRLb8Xxjf/BgaT8ituQ11xjXA/9/F1OXWVN04zvlk7r39PvRyJQFL0tf+766kkjSW3zW72yrDEqQswnjwE4FUFbOICiCJqbmykWi2zZsoVsNms1w0WjUasZbt4C1TRdzsC1+il1a4FZql2IQf3Sl76U8RoeEsDp06fvkFLeByCE+ABQAb6wwGFulFKOCiHagG8JIc5JKX+wPt9gPi5J2DLvJAw1MdN4mMroU1NTdVvyFzIeZVVlKpmrei1lhBi+piba2zuA6kVe72knjDKLGbtXKird3d0EAgH27d1nq1POzWYVmA1zospwmNWQWrMBEJ+dYWR0lH379uH16t/RA3QKQXt7B1JqJBJJZmd1KrXP6yUa01XIzaFW1XmI+TArRmboNDIyQj6X4/DBQwi7eI4QyOffSOX5N1oNd1YHq7F/0zgpmobrX/8Vx6OP6ce9+mryv/02VJzmllaexIT9euuhnHnVqqFZBsY+F0fQHPJVif1omkZTUxPRaJQtW7agqqql3NXT02NNhWtubiZ47724P/lJRLlM5aabKH7kI3o5/CJiKeX0//3f/13oLdNw/BpwG/DzC02Dk1KOGv8/KYS4B7gB+OkxHvVgdta63W5rQLXb7eb6669fsCW/nkjxRDxXtYhM4Z59+/YyODRHRDMXhPnvWiiKgqqpaFIPm3q6u9m0aTPNLS3MjVI0t9XzI7qB0L2LimHYFEXU9Tf0GTGjpFJpDhw4UPU0shap0Ks+0WiUWLMxGyWXJ55I0NXVDUAkEiEajdLU5DPyHHP7ANNw6H9pmq6R4nQ62b1nr8WWnb9+58Ic0/gpxiK2qiUPPYTj4UeQkYh+jKeewv3gg6ivfjWg56g0VaNYKqBp+pAvhxHeaEZ4qtS59tWJ1TnlepdDoTlQHZbUJkwdDgfNzc3W7J5cLsfMzAxjd92F/+MfpxAM4giFcH3/+7j/6q8offjD8373jUShUFh1aGXomL4PfXp9boFt/IAipUwb/34Z8KerPd/l4LIxHqqqks1mOXHiBNu2bVuUUFOPG1IsV0jkCkip31iDA/0US2UOHz5szHmpNjbmupmXM0AvB2qaJJnUqzJ79uzB7w8YN3z1atNddGmUdqWlqanrhc53wDVN0tfXh0MR7Nu3TxcVslkzRdQklI1F7BBCr+w0NbF58ybK5TKJRIKRkWHyuTzBUJBoNEowGDI8uLkYoVJR6enpJhwO09nZOWfQjGSxmTuxGxs7zMY/M62hdJ3XBZwVo0PF60E5d17v0gVcLhdFtUhvX59F81dVDdCs7lqh6OGNXalsMV5HrcTgUlULs/rhymRwuVyUfT4q5TJFhwO+/W3GfvM3aW5uXvMgp+VijUnYf0B3Sr9lfOcfSSnfJoTYhC6KfCvQDtxjvO8Eviil/Mbaz3xhXDZhi87QHOWKK66YY1MugHphy1hcH1RdLpct4Z7tO3biUKCiYty89vMwDIblThuxtkHuiMfjlEolqyoDc4OP7KQovQtTWobDMkZGAtAqq1ZUyoUCPUODRKNR2tvbqypP5oJeuBIljH3qJ+9yuWhtbaWt1aSCp0kkdK6Ey+UmFotaeaKurm46Oztpbo5Z52bscl7upB4stXSMpd3ZCU/+BCE1a/yltmlOzT6by3GhR6fph4JB3Zsyc0jS8MxUPbwRQhghzlzOyQ6nQyEWmL/Al1uqla2t+vVyOHA5nVAqUd62zRLVLpfLVUxRWH/y2WpU02s+v2eB10fR2/ExdE6PrOlAK8Ql9zyklKRSKRKJBEePHl3W8Kda45Ep6AOTs9nMfOEeaQ6NrhrFbPy/LYY2kolCSmZmZtA0yRVXXGHdoLW9MGbZVhGCUqlEpaIarvncPs2bxvmNbyC+/BXUfJ69Vx/B9Y536DKBJsxFDGBWUWqvk5UJAKz39RBKUQThcIhwWG9QKxQKJIzwJpfN0tLSjMfttiVW61xUW05CNY1gzwWUJ58Erwd50wshFkUCldtfifvUKegfAIeCtnkT6mt+Sf8tMln6envZtXs3fn+THvoYxsE8kMMo9+p5IWmFOZpRCtYTrPp1bw3N9zrMa7ucRV657TacX/saypkz+hPD60X94AfZunWrJRwUj8eZmpqymKLlcplCobDuXslGlZYvFS6p8TAlBwF27dq1oqlxduMxkcgyNT01T7jHiqHFXFLT/J96D4NKuUJXdxcut5tgIFA34WeHqkk8Hjcup5MzZ87Q1OSz2r3NmSPi5EnkF75Ixu0m0N6G48xZKl/4Iupbf716Z/ZEITXhBDXBTz6PdLn1ubKC2nfx+XwUiyWknOLQ4UMUiyUmJifI9mbx+ZuIRfVZIs56YxwNw+Q4cRLHX/4lSrmMBshvPET5Yx+DWBT8TZT/5E+QF3p1D27XTnC5SSSSDA4OViWAMUMhaZLX9Jd1b8Oo9Jhem6bOeYhCw6EohH0L3xPLWoxeL4XPfhbHj34E+TzaNdcgjVGgUM0UlVKSTCY5d+5clVfS3NxszV5ZDVZJDLvsccnClkwmw4kTJ9i1axf5fL5uAnSpfQDMpvOc675ALpczPAVzfOH8WbCGc6HXSGo4HLmcLje4des2Y0xDwdgPzE2zt+1LSqSmglDYuWsnIMlmc8Tjcc6dP49DKESiEQJPPU1TuUygtUXnj4SDOM6cpiJtRRvms0LNEEXaw4lkCucnP4nS1QVOB5U3vhH50pda10SvqMD09BTj4xPs338At9tFIIAVsmTSGRJJUxXMoSddI1G8PttTVoD44hd1Lyga0UOT2VmUH3wf7dgxfRuXGw7st8zW7OwsY6OjHD54EEe9Kff2EE3YksxG3sXhEDiMLlnTK2kNNSGlRrms3xuKohhhzgoXsduNetNNS24mhMDj8eD3+7nyyistr2RycpLu7m58Pp+VlF1JV+1GeDGXAy6J5zE+Pk5vb6818HpwcHDFzXEAxWKJ7/3oSTzeJvbv14V77N5FvfqlNYPFRkmfnZ1leHiIPXv20tTUxOzsjEX8kkZjS3XJU9NjbmF3x/WEZiAQYNvWLeQLJfr6eiloKtukRqFQwONyoeSLyM5NVp7BfNLbk6YmNAnCRgRz/Ou/oHR3I6NRRLmM43OfR9u6DfbvQ2ayMD3FZLFEQtOMKs6cZyHQzz8QDBAIBtiyZQulUpFEPMHA4CClUpFgMGQkXYOIfB7hdKAJBSGlPky7YBrU6grS+PgE8fgs+w8cNJK183ylqk+YlaGFHsaKInA73XQ0h6wSs6qq1gOmUqkY3sr6U9Tt+6z1SswKjr2rdjleyU9jOz5cAuOhd1KOcPToUYur4HQ6dfWuFUBVVb798GPEmttobm6xXpeAMDJ79Vq67aVAVZOMjo2SSsQ5eOAQLrd+PsKWI1HQp6zpyVXTcEg9Jq9z80sJFVVlcKAff1MTW37lV3GOjyPPnqMiNcoOJ+MvfBFN0zNEImGcTicKApX5eQ57ohJAOXsWggF9abrdiEwGpb8fWSzi/OQnKOXztAPtb3+7vvBN7+ZCL2JiXO/m3bXT2r/b7aGtvY329nYqqkY6nSIen2VgYIDNBw/Q9p3v6AxVTUVzutCuvdZ2lbF4I7lcnv37D8wZUikt1TSqPmGD0HlbemWKuTyLgdZwk0E40180OUCapjE0NITH46mSc1i1V1KDhQySMKpdptZHpVJZtleylqa4yxkX3XhEIhGuvfbaqtBjpcOuJycnSWeztO44iNdXX1TWiq2tykg1j0DTVC5c6MXhcHDw4CEQirWtxd2whT7SZjh0Nnb96kSpVKSrq5v29jZaW1oRQlB5z3txdnehFAq4du4k5nSSSCQ4d24ch8NBNBohEoni83rnJUbtFkVraUFMTqE0KXoYJQTS58P5yU9QKJZw+v04EfCZz1A+cAARjeK87z4cX/4KmsOBU9OovO61aLffztzuddq80zEnzyelpLB7NwmPB88jj6L5vGR+8Rfxbd6CHwz2mEZ/fz8Ae/fuqY7pbdd88SoOFn9EMOevuByOuhUWgMHBQRKJBEeOHMHUGzX/M3MLazEkyxU/djqdNjnG+V6JSVALhULkcrlVex5CiA8BvwFMGS+939A1rd1u0Zm2G4FLErbUJo+WazyknBsGVRJuvF7fPAe5Xru4/ln9qYJQKJVKdHV10dLSQoetAc2sophdmyaxoYoxKszqjbSYqObhdBnDC2zfvt2qfEikPnfl0CErzg8Awb4+tg0OUY6GmYqE6e/vp1IpEw6HiUajBPyBeZ5N5Td+E/ef/zkyk0VoKvLaayhv3YqazeIIh3G6jORiuQRT06BqKF+5E/xNKE4nmqri/PJXKD3/+RCNWuVpbNfQ7IXxNTXBm98Mb34z5XIZkknGxkbJ53L4A0Hy+RyhUJAtW7ZZ3I16LFqT6iKUanLePIKamOPetIZ88+4R87fPZrNcddVVc1UwxcyTaFYoY/5XqVRQFMX6bzlYTSi0kFcyPj7ON7/5Tf7jP/4Dj8fD2NjYiga02/AJKeVfL3J8B0vPtF13XPJSLSxv2LXJPPV4PFxx1RF6vvUwmtRQhEN3dw0+RT0dTNCTcZqU5LMZunt62LlTlwqsq71hhi2G4VBtjFGwUcLNKoIQet5kZIR94TCBv/97xPAwctdOKm/9DZRYzEbEEogvfxnn3fcgpMTpUNjyvOfR8bu/i6rp2f6JiUl6Mxfw+wNEotG5ysie3ZT/6q+grw+afOS2befCmTNc7fagqBWky60bDoCWFkglEYpAM8JD4XTqPJFUGunzoXzjGyiTk7phu/FG3ZNBGo1upvcmcblctLS00NraSqlU4vz58zgcDhKJJNnseWKRMOFIBE+dpKBhj+eo+kYVaaFeHKfDQbTG65BS0tXVhaqqCwpl2/MUgGVAzNBmueHNes2pNb2Sffv2kclkuPvuu/nVX/1VXvOa1/Dbv/3ba9p/HdzA0jNt1x2XhfFYbF4t6E/048ePW8zTgakkikNBVTUUxWGFKA6h5w5quzjNRTs9Pc34+DgHDhzA5/PqTyrz7rZBMSofqqaCGabYeljst7xEMjY2RiIR59DOnXj/8P9CIglNPpTjJ3B9/ON6idPp0PkZX/oyrs/8EwoC6fejbd6M8tiP4LbbcezYTiwWpcWgo2ezWRLxBGOjIzhdLqKRCOFIFO9115JOp+m7cIHdhw6h/sH/h/Kpv0OkU0gEld/8Tb2k6vEgy2XEyCgyEEA6FFSfDxGL4v6TD0Ffr15J+fa3UfsHUH/1V6zvZPHIjC+sSaiUS5w/30VHRyctLToNvFAokkzG6evrR1UrBEIhYtEogUDAuo7UXC97WFMLM9dhbS8lZ8+eNcLL5Q86N70Np9NZ5ZWYRkRVVcuI2I3Fes9sURSF1tZWXvjCF/Knf7pqtvjvCSHeBDwJvEtKGa95v9682tqZtuuOy4JhuljYYnbWmszTbKFMKlcyjEY18Uu13fC1HbHFYompqSlLKlBKs9t1vudhaquaYYbAJl+oWbtF0/QGPik1Duw/gOjthWQKJRhEEwIZCaOMjsD0NHS0I555Bue994LDiVQUXct0chItHEYU8phcU/24c7J3ZmVEFzHqo1DQe0Z27NhBU5MPrrmW0t//nR6qxGKISASpabj+3+ehVEKkU4hEHNnSQulDH0b09eljKUIhPdyQGo7770d97S8jDDKZdVWNsKNcLHK+u4utW7cRiYSt970eD962DtrbO6hUVFLJJNNT0/T19uH3+wlHIlZi2IRihDi1oY7b6SDqn0s2aprG6dOnaWpqYteuXavmSti9El1iQDNU3XWv0nxwmffhxZ5Tu1BH7Uc/+lGATwN/hm7O/wz4G/S2fDvqXZi10VqXgUuW87DfoPXCFrMFfmJioqqzdjyRRaAnNe2Ucztvw54srVQqdPf0ALBr126bxmitGrp+O0up4XQ4qVQqnDlzxhJ18fm8OBS9J0PvtK3Q09NDKBRmU2en/vN5vbrkoUn0qlT0czEIU0pPDyhCnwpXKIDDCekMor0daWiVmP0j1RcMPF4v7W3tSCmJx+O0trYakoxD+P1+otEY4e3bdYV0JMr0FMp3vouMxpDNLaBWdGX0QAAlEdfPw2BxCglCalBRka7591w2l+XChQvs2rmLYDBYpSNiVyhzOh3EmmPEDE5JLptlZnaW8fHxuXkp0Qg+r6+q01iXMJS0hua8Dk3TOHHiBJFIZFG5ydXA7m2YXolZCs7lcng8nqp8yVqx1NiFRTpqkVJOmP8WQvwr8PU6m12SebWXZdiiqipnzpxBCMHRo0etHzCRLZArltGTmg5LmMakaddm4PL5Al1d5+ns7DQ8lbmFMV8N3UycSISicODAASqVij6HZHiYQiFPKKQnM91uF70XLtCxaZM10hCAzZuRz30uymOPWtlB7eUvR0SiesnV0LsUW7Ygx8cRmQwyEqH0R38ETX4EVPXHmNCf/hr9A4NUKhV9qJXioLm5BSF0RTB9tqwu2xeJRGlJp3E59XyQkBqawwkOBXI51L37UPwBRCqF9LihUEC99jocfh/q3EBYAFKpNIMD/ezduxefz1fVC6MZvSr1nnsCaDKSiGwVFEslEokEo0ND5ApFQqEgkUiUUCiEooDblutQVZXjx4/T2tq6IsWt1cDulYyMjJDJZNi+fbt1HutRCs5ms1a370ohhOiUc7NnXwWcqrPZEyw903bdcVkYD/sEuEKhwPHjx+ns7GTr1q3Wk0hKyXhiTnpQKLpIsSLMAcjVd3AqlaKvr5edO3cRCoVIplJWmCPqJEml0bAlkFanq5kobGlpAU2SSCUZHx8nmUwSCgV1YWRVA0XRF7wQVH7ndxDXXosyPo62dQvy+usxKefa85+P9thjOl8jGkXr3ET5j/8YthgdxELohKza85MaXT0X8Pm8bN++y3ram7SKYDBoaEVspVAokkjEuVDIs9PjxRNPoAT8OIsFZEsrYssmpNNF+c/+FOd//idiYhL18CG0X/1Vnagm5ybbmWLT+/btx+Nxz7teimFEajkaUFvFkXjcLtpaW2lrbUWz+pniDA0N4vF4ObC9k2LRj6IolnLcKisTq8LIyAgTExNcffXVVUnXWoKaGdasxJCspVQL/KUQ4mr0S9kP/BaAvaPW0DadN9N2tQdcLsQaOv5W/cFyuTyPjv7oo49y6NAhTp8+zcGDB6uf6MBUKlc15X5goJ9wOEI0EkZSLf8/OTnJ+PgY+/fvx+PRn2a9vb20tLQQDocMNufc3a5pqtVKb/925tPVLGnqCdcJ9uzZQ6VSId3VBU88juJ04nj+8wnu2GktsvkcSx2KqiLPd0GpiNy1G0ILC8QIoT+xL3R3E2tuob29rf52Nccy/1bHx+Hv/wExMECurZX4G36F4O6dhEJhQwlt4X1MTU0xNT3FgX37521rbS/nPmgPtxYq25qlbTvfVEqJVikTUkpMT0+TTqdpaWlhx44dOtP1IvSEDA8PMzk5yZEjRxZNlpq5EtOomDA/s5Ax+cAHPsArXvEKbrnllrrvL4GNvwCrxGXheYA+M9RUmK4nmjKbzlf9rSgOXf7PuGs1w4MeHOinUCwYA6rnbgSHwzbfVujGRtVsxC9z9Zg/lRGTC3SjNDw8TC6Xn6N9D0zQ8qlPoOV1DZHy979Pz2/+JsVolEg4TCQao6lpPl9BczjA68V1/9f0EOKFL0J78YvqcrXz+TzdXV1s277dEgSew9xyr+VomIva0dEBH/0IAH4pkZkM8XiCoaER3B43kWiESDiK1+PW96FqkM8zmkySSqfZbxgOe5Ne1fFNIWaLo2Ecv47hMBm6ZiOfdbmFYHtHMz6nbvQPHToE6GSwdDpNOBympaWFWCy2rCHoK8XQ0BDT09NLGg6Ynyuxl4NhYa/kp3HUJFwGxsPUVahUKrzgBS9Y8Afc0xllKpljJp3XtS4NY2A+AFS1QldXN36/n/379+vCPbbPC2Fqjeq3uapJpCESZP7Y9bwwTWrWYKj9+/Zai9z5la+glspgtP67E0n2nztH8f/8HxLxJKOjI+TzharY3ukQaEMjuP/4j5HlEjgcuE+fplzIod76iqrjZjIZent72bVrN4GA31hscwaj3rPdrGAIMf89IRSCASO82baVQqFAPB6nr7cXVa2w6dx5Wv/7v5HlEm1t7XR+9KMoTiem+rmZ55iTPaxjIMxEsWFY7JdzboiTua3+P163E48iefrpZ9i/f78lpWDOjk0mk0xNTdHX11fFN1kPwePBwUFmZmY4cuTIinMZ9ZKutQbFvK9yuVzDeKw3zMn2LS0t+Hy+RX9Ah6LQEQ3QHPIxEc8ybsuT5PN5uru72Lx5M83NLVbOwCqvGsZG/3eNxqjRPyE1iRTVPmKpWKKnp5vmllYrZDD5HkomjXTOielKh4JIp3E5nbS0NtPS0owmJelUingizuDgIF6fl62PPIKroM+BBVALRRz3P4B2623W0z0ejzMyMsKBA/txu/VKjblkTfNRLyTSPQ4zC4zNm5FVYQWA1+uls7OTzs5O1J4evP/1/ygKgXS5cU9MoH70I1Q+8Unruun7Nxva6hO85ozLHHnObCak3rgIAU1OyTPPPMOhQ4fmiUAJIaom2ufzeaanpzl37hzFYpHm5mYjFF15u/zg4CCzs7OrMhy1qEdQM8ObSqXCk08+SalUWmwXz0pcslJtKpXi5MmT1mT7ycnJugrqtXA5HGxpCVHMhBiP5yypwN279xAIBIwwQ9/WZEgqQi+xqhV1HtXc7MQ1DYfpoufzeXov9LClhtdgdmKoN96I8+w5NKdTr9CoGpXnPGcuDyD0p204HDb6RTSyuTzlikqlXKZSKOi8A01DKor11J6anGRqekYPj+qMrrQiK6s0Pbcoq8IKIzwwOFlosv6cEwE4+vqoaBqOpibcDicVtwv3hQs8deoUbq/X0kp1u11GOVvW9S5quxBNL8WmpV6FSrHIwMgQV1555bKezD6fr0rEZ3Z2lrGxMc6dO0cgELCS264lFNIHBgZIJBJVNPf1hOmVaJrGu971Ll74whdy1VVXrftxLjUuWUt+d3c3V199tZWFNgk6y2X3NXk8+OQsqVSCa49chWreovMylYakv5EjqaiqMaDaYIyK6nteIkkmEgwODVgt+rUQAtSX3ozM5XE++ACa4kB9wxvgeTfWHtzKnQAEAk3I216B6wffx5nNoQEVtcLQ0VuQQ0MUiyU0TePAgf04FQVt3pHNc8Q6af38DdKVPWdjO76+zkWVBID+WUG5XGYkk2G34kBxOJFC4CyVkbFmrjxyFfl8nng8QU9PD5qmEYvqLNemJp8RIgkjlVG3s6UmqWp4JkIPy7JTIzz3+mtWVYlwOBxVjWmZTIapqSmeeeYZACu88fv9VUazv7+fZDLJlVdeuWET50D3Pv7wD/8Qp9PJpz/96Q091qXCJam2zMzM4PV6qxJgTz/9NPv376+7WGuhaRpPPfUUhUKBn/u5n9P7LLIFJhJZSpV6S05jcnKaqalJOto7iEYjenZAzLc14+MTzM7Osn/vXpzmeIOap/u8SyZBcZhT5uaj9hjK6Cji3ntRslkqL7iJ0nXX0tXVTaVSQQhBMBAgaiReUcSi+zKPX5dctsAnFCGoqBqVcpmu7i42d26i9T//E+XRR8GhoEgofuADcPURc/eATrhLJBLE43Hy+YI+UzZq5nMUjFafmqPXmBUJqXSK0eFBXvHC5677sCbQw+Hp6Wmmp6fJZrNEo1FaWlpIJpNks9kqecmNgKZp/Mmf/AmpVIp//ud/XuuxLttqyyUxHnZKsIkTJ06wc+fORWdbwFyexOfzWf0O1glJyXQ6z1Qyh2qK+RhEpoqqz0CNx+OkUil8Pi/N0QihSBSnkRgcGBigUlHZtWtnlV6HvXt2oRKsuUjmPd0XfCbPPfm7u7uJRqN0dHSgaZJMJs3sbJx0OoXH4yVmtOy7XK4FKxkW4xNhMVwXQz6f50JPD9u27yAUCuqh19lzkEgg9+yGNntZ2F6K0o+vadIgp8XJpNO4PR4ikbBeFXE45275Gm/IzOe85Oeuoy0aWuIs1w5N00gkdM8pm80SiURobW2lpaVlQ9S9pJR85CMfYXR0lH//939fjz6ZhvGwo57xOHPmDJs2baqaDFcLU7pwz549uN1uRkdHrdJe1f41jclkjulUTh+RaLTP2wlnuZwuGZiIJ3A6HRRLZSKRCNu2bdVb7mu/nqwZNr0EzNyJsoBHIAQUCiW6u7vYtGkzsVi06lgI/Tzzeb0qkkzE9RyKQZe33/j1DFTtsGk70uk0fX19ep7I78fuWy30Of3SzRlR61SNP/L5vOWVgCQWiRKOhq0h4AAzMzpV/aorDnFwa+siV299ceGCLlN5+PBhK+k6PT2NqqrEYjFaW1sJhUJr5pRIKfmrv/oruru7+dznPrdepeWG8bCjnvE4f/68lT2vh6mpKbq7uy3pwnQ6TX9/P1deeWX9k5OSfLHERDxDMl+ad2OYi6RQKNLV1YXf30SpVEJTVSLRKNFoDJ/PazM4c+MZap/uCy04xZaPqD43XTP1woVedu60kaGsMuh8CCEoFkvE47Mk4gmKpRKRSIRYNKrH9XUUxucSq3N7TCSSDA8NsmevTaTY2lrO8xRq3p2Xu6hnuPSZMkkSiTiFfIFAMIiiKGQyGfbv38+ujiihpuVrgK4WpgZIPp/n8OHD8+6BcrnM7Ows09PTpFIpQqEQLS0tNDc3r3jhSyn51Kc+xdNPP80Xv/jFJZO2K0DDeNihaZouMGNDT08PwWCQ9vb2qtel1DtXTSKPqbCezWatpGst7HRiRVHIlyqMxzNki2XrxAUml6LPWsBIUNUys/EE8XiCYrGgs1ijEYKB4LyfURHCOI5YYMHNdY/qx9UvWSqZZGBwkL1791a7zjULc+4azOeQqapGMpkgEY+Tyeo8gmg0QigU1kdAMJ88NjU9YzBvD1gzamvP16SOyqrX6+dZFKGPX6n3wDbL5ZomGTSqG4qiEA76uWbfNlpaWpatlr8aSCm5cOECxWKRQ4cOLelVmCNApqammJ2dtfRLW1tbl8zDSSn59Kc/zcMPP8xXvvKV9f5eDeNhRz3j0dfXh8fjYdOmTVXbnTp1yspt2BNPhUKB06dPc911182dkI2oYw9TTKTzJcYTWUpllemZGcbGRtmzZ6/1BK6apKYIKhWVZDJJPD5L1ligsVhUp3fbnvT1R1jXSx7C5NQUk5OT7Nu3b+Gnk3F8M3eycN7EDCMk6bSZz0nidnusoU/mMcbHJ0gk4uzbu3eZdPPFvQswjaecpxJms1qMjIySzWbZs2cPioDmJieFTJLp6Wlg4arIWiClpKenh1KptCzDUQ+FQoHp6WmmpqYoFovEYjFaWlrmDdGWUvJv//ZvPPTQQ9x9990rUlVfJhrGw456xmNoaAgpJdu2bQOgWCzyzDPP0NnZab1mR7lc5umnn+aGG27QT2YJw2GdtJQ8c/o8/WPT7Ny9p0ZhfH5lwBS00aS+QGdnZ62EazSqL1Cn01nFe7AcB1F93JGRUXLZDHv37rXUuuaOXf/pbr1ZB/bznaOG6zf+7GzcyD9gTa/ft28filAwJmTWeBd1DISR51G1+d5F/arwnEqYqmkMDQ1TKpXYtWsXiiLwuV3s7ohY25tVkampKXK5nLVAo9HommakdHfrlauViActBvtgqEQigd/vJxgM4vF4+Na3vsW9997LvffeuyGVIxrGo+aDUs5j3I2OjlIsFtm5c6dFIDtw4MCCrcyapvHjH/+Yn/u5n7M6H01d0YVuGE3TOHPmDE6nkz179jKbLepJ1SWugdHDivmV9YRrnmQizmw8gcOhEI3G9ESmx219Yu64+nxaxaGwY/t2I79heBeW4NDCuQ6pyXlzXfQToc6tJSzrpbvuvZRKJRwOB+VSkWAoTCxmKH0ZWqxVxZT5e9N3p+hq5/aqTl1vBN0L6R/oR0pdsMj8Pba3hgkuMMRJ0zQr/xCPx/H7/VZVZLn5A1OuUDNGT2xEU52UusLbT37yE971rncxOTnJ7//+7/OGN7yBvXv3rvvxuIyNxyXvbTFhksTMmS52Alk9mL0oyzUcpVKJEydO0NbWZnkybeEmYgEvk8kc8UxhAdfcHG4tbd6EIOBvwu9vYtPmzVYbfG/vBVRVNQSE9ISrqmqGaFCIzs6OufMzFrdAX5Bavae7laE0t53zLuwiPHaYfA8pJT3d3fiamti9excg0DTViOun6evr10WEInpOB6HU9SSkdXyq+lsWKj9rwIXeXtxuF1u3bLG+lM/tWtBwgP572mekmKSvp59+2nrPDG/qwTQcUsoNMxygf/9AIMDk5CStra08+OCDPPzww1y4cGGjjMdli8vG85ienqanpweXy8VVV121rKfNo48+ytGjR4GF26FBT66ePHmS3bt309pav0RYLKuMJ7Kk8/b5MXNPcapf1b9Hnf2olQqz8TjxeIJCIU+lUqG1tY0tW7bUTSxaVRzDE7H/Hgs/3fVqz0L7KldUuru7iEajbOrsNHInNY1pxgKNxxMkkwncxnDscDiCewlZgYVe1zTJhZ4efE1NbN68yVjA+taLeR1LoVgsWuFNoVAgGo3S2tpq5R+klJw/f94KzTa6jf/ee+/l05/+NF//+teXHMq+DrhsPY9LYjyAqiFPqqryk5/8hHK5zPOe97xlZcY1TeOZZ55BVVV9WnxbW13Sz+zsLF1dXRw+fHhJAhpArlhmPJElVywvuHh13U9JvcqItZ9cnp6eHpqbmykUCuRyGfxNAaJGwtVsOKuVZLQM06L5DsMTMkrB9uRI2RAp7uzstEZM6l6TUpc8Zn7a7LI18ySm2LK9XI2xrTVUy5bn0T2sbsLhMB0dHVXHaPK42NUeqfdFVgyzp2V6etrKP5TLZZqamjbU4zDxwAMP8IlPfIIHHnjA6gDeYDSMRy1M41EoFHjmmWdoaWkhl8st2UBUmxgtFotMGRUMuyHx+/0MDw8zNjbGVVddteIseCpX1CszlVptVaqJXyahC3AY5clUKs3AgN6sp2t66E/lTDpNPJEgkUji9XqJRvWGs3qcgjmZxJoqjrAZFuNtYYQ1xWKJrjoixXZlMkXUV/6yo1wuk4zHmU0mKRYKVWMoHcp8Ap2manR1nyMWa6Gtbb5g0Vq8jsVg6pyaXuxKyqurwbe+9S0+9rGP8eCDD65aVnAVaBiPWpRKJeLxOKdPn+bQoUN4PB7Onz/PNddcs/ABl6iomNn7iYkJUqkULpfLavVezRNJSslspsBkModq8EYW8kYkerJwylAb27dvH25zfOW8Ko7OHJ2NzxKvSbh6jISrGS7pnzWPsPDxc9ksF3p12cVAwJYXqOfBWCGSfYM52H0ZTdNIpdLE47Ok02mamvxEbbNkKpUK5893samjg1hzc9W5wvp6HXaYIxlcLhd79ugT6+qVV1tbW9c04d7Ed7/7XT70oQ/xwAMP1DWQG4iG8ahFX18fAwMDXH311fh8PqtnxcxhzDvYMhOjqqpy6tQpfD4foVCIqakpMpkMsViMtrY2IpHIig2JqmlMp/JMp/PzKx42jI6OkUom2bevPpei6vuAYRZM5miceHwWVdU7VyN2hqtcnBqfSqUYGBhgz549+Hw+q2QqFuFoCAHIuQSsdV51ysz69nplKJvV+STJZBJFcVAsFti8eQttra2W1bGf6462MAHv+nodUkrOnDmDx+Nh9+7ddX9PM7yZmpoimUwSDAYt9uhK2Z8//OEPef/7388DDzwwLyS7CGgYj6oPSsm5c+fYtm2b5bKrqsoTTzzBc5/73Lrbm3T2xZ4ghUKBEydOsGXLlnlks9nZWSYnJ0kmk4TDYVpbW2lubl7RE6lcUfXKTLYINZ5EX/8Amqaxc+dOY3hzfbamiVpCmjRYVmbn6uxsnGKxYCm2B4MBfc6MqCZkzc7GGRsdZc/evVUixcL434VyMtVJz7m/FmvkQ+rK8lKTFEslzp07RygUpFAooKqapfthyi/6vW52tq1vQlFKyenTp/H5fMue5WKyR6enp5mZmbHa+VtaWpYMbx577DHe/e53c//997Nly5b1+horQcN41KJWBFlKyWOPPcbznve8qteWQ/wC/el7+vRpDhw4sGgiS0pJIpFgcnKS2dlZ/H4/bW1ttLS0LLufoViuMBbPkimUUFWNCz09NPmb2Lx5c9U52ukTS14sifHZuaWrqhqpVJJ4PEE2k6HJ77cYrk6ng4mJSaanp9i3dx/OhejmwDzRngWgexz1B1PbjUohX6DnQjfbt+0gGNJp/RW1ul0/FApyZM82Nne0rlv7u91w7N69e9X7KRQKTE1NMT09TalUqgpv7L/fk08+ydvf/nbuu+8+axzDJUDDeNRiIQV103isxHBMTk7S29vLVVddtaJEmVmunJiYYGZmBrfbTVtbG62trcvqT4insnz/x08RjjXPKwFXuf+ympdRdz4u1VUcu+Cwvi9JOpNhdlYPGUAPPQ7s34fLvUAy2Eb8qq7J1P/xTLq5OXqh3r7MKtKuXbsIGJTyWnFkTZOoxRw+qVdwAoGA5emttmHMnB7n9/vZtWvXqvZRD6qqMjMzw9TUFKlUimAwyPDwMIFAgPe85z3cc88963q8VaBhPGqxmPFYCdV8YGCAmZmZZXNDFkM2m9VHDkxNWTNGFyoB53I5Tpw4wd69e3F4/Uwmc1WVmQVb26nWV10QNt6H/amvf+dBisUifo8Hz513Ejl3FtHegfa2t+HeZk6eq89ItYdINW/Vp5sjrSpSJpOlt7eXPXv0KlJVuFNzvJ1tEfxel2WgJycnq0KGlYgYmz1OwWCQnTt3Luszq4EZ3nzwgx/kvvvuY//+/bz2ta/ld3/3dze8BLwILlvjcckYposZhOVSzc+dOwfANddcsy6usd+YcLZjxw7LtT19+vS8EnAikeDs2bNcccUVFnck3OSxKjMVTVuY8i50U1A7zmCeN2BnlQo9N6Jpeou52+1m3769uP/qrxDf+x6oGnJwCPXUSc584I8IbtlMJBKdP/rBdpDa4yvMz3WYuqpISKfS9A/0s2/fXCewvWxssmUlEPC68XuNSpMQ1lCq3bt3W9f17NmzlMtlq7S60IwWTdM4efIk4XB43cdO1kIIwejoKD/+8Y/53ve+h9/v59FHH72UhuOyxiXzPCqVyrz5tI8++ijXX389S03jKpfLnDhxgubmZrabvSIbiHK5bHFJMpkMqqpy6NAhWlpa5h1b1TSmknlmMgtUZmpCCRDWIl0ssSo1jfPd3YSCITZt6oSKiufWW8HlBGEqw0uK73wHM9dcYyVcg8EQsVhMT7jWjpeQGD0rcsFrqAjBbDzB8PCQUX42w7mFMzm72iI0eZf2AiuVCjMzM9Z1NVW+YrGYJSB88uRJIpHIRck5dHV18aY3vYkvfOELC+rEXAJctpbrsultMVWdnnjiCVpadLJRvaeRGS7s2rXrotXbXS4XmzZtskSMNm3axPj4OD09PUSjUdra2ohGowghjBERfmJBvWcmkS1Y+6ntR6nmRMxfjDoNQ1JRdS5FW1srba1tuv3RZz3OO1eny13VI5JI6u3v/f39BPx+IgZHw+FQLDqJgmJ5OtUQTM/MMDqqT99zuVy2O7l+TSbgdS/LcIA+4Ly9vZ329nZLLtAUfWpqaiKfz9PW1nZRDEdfXx9vetOb+NznPnc5GY7LGpfc86jNb5gJrImJCWtAcFtbG+FwmEQiwblz5zh8+DCh0MbrX5ow27yLxSKHDx+uGvYTj8eZmJgw5teGaGtrqyoB50sVJhJ6ZWYhWFUZIw6wzIkQFApFayaNVUUyQjrl7/8B5/88CKqq+y6xGOV//RdkzRgDnVWqVfWyeDw6wzUcjuB2ubAqy7bPzUxPMz4xyf79++ZVokzV9NrbYFd7hCbP2nJPqqry1FNP4XQ6KZfLG84cHRwc5LWvfS2f/exnF+QZrRWFQoGbbrqJYrFIpVLhNa95DR/+8IeX89HL1vO4ZMZDVVUraWof01e7zezsLBMTuqK5lJL9+/fT3t5+0eJQVVWt8qDJZKwHc7qZWQJuamqqKgGn80XGEzmK5Wr5xXmJTVtlJp/XKxvm3FbrM2Z+QtNwfu0+eOJJaG1DfdMbkTUzfvVz04lbGJKItdqoQlEMbZIoXq8HAUxMTjAzPcveffuqNE/qnrNhdgJeNzvWyOtQVZUTJ07Q0tLC1q168tdsQZiamqJUKtHc3LxuuqMjIyP80i/9Ev/0T/9URRNYb5it/IFAgHK5zPOf/3w+9alP1eU11aBhPGphWuClEqOmnFw6nWbz5s1MT09bRK/29vY1CccshXK5zPHjx+no6FgRQcheYZienq4qAWdLujhzWVUXZHMCZA2JxN17di/raWvPn8x/r0Y0yC5aJLAYrolEnEpFxel06IZ63z4cDgdygX3NfWFdcGhnewSfe/WRsKqqHD9+nLa2tgWvd6VSsQh/5ixbM0+yUqXy8fFxXvOa1/CJT3yCF77whas+75Uil8vx/Oc/n09/+tM85znPWWrzhvGoxdve9jb6+/s5duwYr3jFK+o2GplPfa/Xa6hv6ddRSkk8HmdycpJ4PG5pn67mBloI+Xzeyq0s1Ma/XORyOSYnJ5mamkIIQUtrKw5vkExZWj0zdiQSSQaHBtlniBTXHzQ9H3a9DxOmgZi/LfPStFJKhoaGSaeSuNxuCoUioVCQ5lgMfyBgGOn6idKgz8P21tWHkssxHLUwCX+m7qjP57OYo0vxdCYnJ3n1q1/NX/7lX/LzP//zqz7vlUBVVa677jp6enr43d/9Xf7iL/5iOR9rGI95HzbEW+666y6+/vWv09TUxB133MHtt99OW1sbs7Oz9Pb20tnZuejNZA8XZmZm8Pv9tLe309LSsmpDYrJV681PXSvMUuXk5CSlcgXFF0Lx+PE16ZyH6ekZJifG2VujcboUS7XaIzDvt/k6HlWw8T4kksHBISqVCrt37dJp8JoklUoyOxsnk0kT9PsJRaNEwpGqGbYAuzuiq/Y6TMPR3t7O5s2bV7UPc5yG6e0BFp+kVkBoenqaV7/61fzZn/0ZL3/5y1d1vLUgkUjwqle9ir//+7/niiuuWGrzhvFYdEdS5y989atf5b777rNKo//yL/+yLH0P+37S6bR1A3m9XsuQLJdAZooSrZStuhqY33N0fILRmRRFTU9sHti3vy7d3DQRteFJXUIY9b0LE3aWq9Qk/QP9CKGwY/s2rOypDWbMPjs7SyqpeyaxWEwf/xAKrNrrUFXV0qq19yOtFbUCQs3NzZTLZVpbW/nlX/5l/uiP/ojbb7993Y63Unz4wx/G7/fz7ne/e6lNG8ZjuXjooYd497vfze23384jjzyCqqrcdtttHDt2jK1bt64oQWbmHaampnC5XLS3ty9KPR8ZGbH0PzZyLEAtzGrO+NQMec3BTDJDKBQkGo0RDAYNAeM5QhZUV2YW5IgIkFodEWObg6ITzy7g8XgMtTNRN9SpPX4+nycR1zVct0Z9bNnUQWskgs/vh2V6fJVKhWeeeYbNmzfT2dm5vIu1CpiJ949+9KPce++9XHnllbz97W/nF37hFy7a72zeg5FIhHw+z8te9jLe9773cdttty310YbxWC5OnDjB5s2baW5uRkrJ+Pg4d999N3fffTfZbJbbbruNO+64Y9kdlSZMl3ZychKHw0FbWxttbW14PB4rKWvOMV2vvMlyYOpSKIrC/v37EUKQyOTpHhxnfGqKdDqD3++nORYjGApVhwvGIq/3Q9RLxppaqQ6jd0XT9BEFgUBAJ56ZZqjOfhcyUCGfh3YPON/8ZgLf/CZSCGZf9zrUv/5rAguwRuHiGQ4T6XSa17zmNfzO7/wOO3bs4P777+eDH/zghoycrIcTJ07wa7/2a9ZMoV/+5V/mgx/84HI+2jAe64HJyUnuvfdevvrVrzI7O8utt97KHXfcYS265aJQKFiGxKTDB4PBVc/4WC1M7RGzZ6NK7k9K4tkiE4kMyWSKmVl9JouuQKaHCw6HwzIQK5qRC1RUje7uLiKRKB0d+qCt+ZKIcyZjoR97T0eU4B9/AOdnP4so6IQ4zeej//d/n76XvczqWLXPO6lUKjz99NNs3br1ouhjZLNZfvmXf5m3vOUtvPGNb9zw460zGsZjvTE7O8t9993HV7/6VcbGxnjZy17Gq171Kg4dOrTs0q359HO5XBZ71N7DspGoVCpWknCxhLCmSabTOaZSeTRNI5fL6yMnEwnDDY4Si0WtnE6t8thCx+7u6qalrYXWlqUrSfO1UnWEmjxsawnhvf56lLNnq4/xildQ+NKXrKpYIpEgGAwSi8UYHh5m+/bt86YDbgTy+Tyvfe1ref3rX8+v//qvb/jxNgAN47GRSCaT3H///dx999309fXx0pe+lGPHjnHkyJEFDYkpHGS/ic0E5sTEBKVSiZaWFtrb29d1mhnoybzjx4+vaAGVVY2pZI7ZTAGzHFu0DXcSQhhT4qL4fJ66+h0CKJXKnO+yCyQbadVl6HjUGqa9nVE8LieeV74S5TvfQRifly4XlV//dcp/8zfWfqSUzM7Ocvr0aRRFsdr0lyt/sBoUCgXe8IY3cOzYMX7rt35rQ7zKoaEh3vSmNzE+Po6iKPzmb/4m73jHO9bzEA3jcbGQTqd58MEH+epXv8r58+d58YtfzLFjx7j++ustQ5LJZDh16hQHDhwgEonU3U+lUrFKqvl83qLJr5XVaPbm7Nu3j1gdNuhSKJYrjCdyZPKlqrCkVNKHNsfjcVS1QiQSpTkWw9vktUSBKuUy586fZ8uWLVXfe444Nt+7mPeKwVYN+jxsbdErLKKrC++LXgSG2puMRCg88gjY+DHmhL8dO3bQ1tZWJX8ghLAMyXpVuEqlEm984xu5+eab+f3f//0NC0fHxsYYGxvj2muvJZ1Oc91113Hvvfdy6NCh9TpEw3hcCuTzeR566CHuuusujh8/zk033cTWrVvp6+vjIx/5yLJDE1VVmZ6etro/Y7EY7e3tKxZWTqfTnDp1al16czL5EhPJHPlS9dhORUCpXC1lGI1EaPL7GR4aZsfOnYRCtSMo5hKli2ml2rff2xnBYy8nT0zg+Pa3welEffnLwfb9TMOxc+fOuoS7Wvr5Um36S6FcLvPmN7+Z5z3vebzrXe+6qHmsO+64g9/7vd/j5ptvXq9dNozHpUaxWOQDH/gAX/ziF+no6OD666/n2LFj3HjjjSsSEdI0zWojT6VSRCIR2tvb5w1ArkU8Huf8+fPrzh9J5opMmCMiajli6FKGU1OTjAwP4XS6CIUjRot+/RKwiYUa3wAifi9bmpeegQO6B/DMM88saDhqUdumb3YtL3V97Z9/61vfypEjR3j/+99/UQ1Hf38/N910E6dOnVrPxs3L1nhcNi35Gw1V1Uctdnd343a7+e53v8tdd93Fe9/7Xo4ePcqxY8d44QtfuGT8bSqMtba2VnXVnj9/nlAoZNHk7Tf65OQk/f39XHPNNes+RT3c5CHkczObKTBlCBHZUSgUmJqc5MDBw/h8PlKpFPHZGfoH+gn4AzTHogRtQ6gsGIajXhNca2h5xs80HLt27aKlpWVZn6lt07df32AwaNHP65XTVVXld37ndzh48OBFNxyZTIZXv/rVfPKTn7yoHd+XEj8znsdCqFQq/PCHP+TOO+/kBz/4AVdffTXHjh3jJS95yYo4ALXCysFgkLa2NorFIhMTExw5cmTNMolLQdU0plJ5ZowRERmjuc6u/mU/32wmzcxswlYCjhKJRHA6nTWJ0rkfe7leR6lU4umnn2bPnj3rMiDJlAicmpqqqzerqipvf/vbaW9v58///M8vquEol8vcdttt3HLLLfzBH/zBeu/+svU8fuaNhx2qqvLoo4/y1a9+lW9/+9scOnSIY8eOcfPNN69YWDmZTNLd3V2VI1mJQvtaUFY1zvePcPJcN/v37cO9gLdjNdJJSdZeAna6iESrS8CghzJ7O6O4nYuT6IrFIs8888y6GY56sCdcP/KRjyClZNu2bXz2s5/dsC7repBS8mu/9mvEYjE++clPbsQhGsbj2QZN03jiiSe46667+OY3v8mePXu44447uOWWW5aceWs2/VUqFQ4cOFDVsOXxeKwn5kZ5ItPT01y4cIEDh64gka+QriNEVEsis3sX5tza2Vm9BKyPxYzR2Rxm8xJeh2k49u7du6pq0kqhaRrvete76O/vx+l0ksvl+M53vnPRPI+HH36YF7zgBVx55ZWW0frYxz7Grbfeul6HaBiPZzM0TeP48ePceeed/M///A9bt27lla98Jbfeeuu8Uq+madY0s3riQdls1uq3cTqdFk1+vbgOExMTDA4OcuTIEWufmUKJ8XiWgk2IaGHN1OpmulKpbE2z2xR00dHeZpHoar+bOXd4tWXolULTNP7kT/6EdDrNZz7zGRRFoVQqXdS+pIuAhvH4aYGUklOnTnHXXXfxwAMP0NLSYmmSuFwuHn74YQ4fPrws3c18Ps/ExMSyRj0sB6Ojo4yOjnL11VfXDY8S2QITCV2IaCEI5pTH7CXbqN9LW8hrlaxruS+mx7F///6LMj1eSslHPvIRxsbG+Ld/+7eL2o90kdEwHj+NkFJy/vx57rrrLu655x4mJye5/fbbee9730tra+uq+m2mpqbQNI3W1lba29uXPdtkaGiIqakpjhw5suhCklIync4zlczNGw+xmGbIvk2xqlyHqTVrjvAsl8vs2rWLLVu2bHjOQUrJX/7lX9LT08PnPve5DcsjveUtb+HrX/86bW1tnDp1akOOsQw0jMdPMxKJBDfffDNve9vbmJ6e5r777sPj8XD77bdzxx130NHRsSJDUiqVrMa95fTb9PX1kUqlquLupVBRNaZSOWbSBt1d6gSzej9qLOBlU6x+riOfz1vdsdls1uphMYWg19sjkFLyqU99iqeffpovfvGLG1rB+sEPfkAgEOBNb3pTw3jUwSU1Hu95z3u4//77cbvd7N69m//4j/9YkC5+OcMUMzLnp0opGRwc5O677+aee+5B0zRuv/12jh07ZmlmLBf2mTGFQsHqtwkYCukXLlygUCisqCHQjmJZZTKZJZUv1Z8zg2DfpvoVlnw+z/Hjxzl48KCluGaWVE1lN5/PZwlBr3WhSyn59Kc/zcMPP8xXvvKVi5Lb6O/v57bbbmsYjzq4pMbjm9/8Ji95yUtwOp28733vA1iuruOzBlJKxsbGLE2SfD7PK17xilVpklQqFSvnkM1mEULg8/lW5HEshFyxzEQiS7ZYTXePBXxsigXmbV/PcNTCVB+zCzKZlaaVkuWklPzbv/0bDz30EHffffe6k+0WQsN4LIzLJmy55557uOuuu/jCF76wnru97DA5Ock999zD3XffbWmSHDt2jH379q1IbvH06dNUKhUcDofFJTFp3GspU6bzJcYTWYrlCkII9m2K4aphn5rNfYcOHVoRmzKfz1uGREppVZqWk9f5z//8T+677z7uu+++iybgAw3jsRguG+Nx++2389rXvpZf/dVfXc/dXtaYmZmxNEnGx8e55ZZbeNWrXsXBgwcX9CTMoc+BQMASENI0zZpvY/bbmFPsVuORmEJEFVWjLVxNjstms5w4cWLNzX1mM5yZ12lubl5Q/uALX/gCX/rSl7j//vs3XFe2Fg3jsTA23Hi89KUvZXx8fN7rH/3oR7njjjusfz/55JPcfffdP7NDhROJRJUmyc033zxPk8QciBSLxRYsBZtjG82xFPWm2K0WpuGwD/heD5TL5aoSsCms3NzczN13382///u/88ADD1h5nouJhvFYGJfc8/jc5z7HZz7zGb797W9f9KfK5Yp0Os0DDzxgaZK85CUv4WUvexlf+tKX+MAHPmBNUlsKJk3enLgXCASs5OVKqyAbZThqYZaAH3zwQf7iL/4CVVX5x3/8R2699dYN7w2qxetf/3q+973vMT09TXt7Ox/+8IcvhRpZw3jUwze+8Q3+4A/+gO9///trHqz004p8Ps+dd97Je9/7Xnbs2MF1113HsWPHeO5zn7siA2COpZiYmKiqgrS2ti7Jk8hkMpw8eZIrr7zyoj39H3jgAf72b/+W973vffzv//4vV199NW9961svyrEvMzSMRz3s2bOHYrFoNU8997nP5TOf+cxad/tTh3e+85285CUv4ZZbbuFb3/oWd955Jz/5yU943vOex6te9SpuvPHGFRGlzCrIxMRE1TjMtra2eU/3S2E4vvnNb/Lnf/7nPPjggxvWWPcsQsN4XGzceeedfOhDH+Ls2bM8/vjjXH/99Zf6lFYNc56vHaVSydIkeeyxx7jhhhs4duwYN91004r5D/Zyqn0sRalU4tSpU1x11VUbLght4rvf/S4f+tCHeOCBB2hra7sox7zM0TAeFxvmLJTf+q3f4q//+q+f1cZjKVQqFX7wgx9w55138sMf/pBrrrmGY8eO8eIXv3jFZU2znDo2NkY2m2Xbtm1s3br1opRHf/jDH/L+97+fBx54YENHMnzjG9/gHe94B6qq8ta3vpX/+3//74Ydax3QMB6XCi960Yt+6o2HHaqq8sgjj/DVr36V73znOxw+fJg77rhjRZok5qzegwcPWuM7VVW1PJKNSGw/9thjvPvd7+brX//6qufVLgeqqrJv3z6+9a1vsWXLFo4ePcp///d/r6dgcV1PcQ24bI3Hz4wM4c8KHA4HN910EzfddBOapvH4449z11138fGPf5w9e/Zw7NgxbrnllgXzF6bhOHLkCE1NTUQiEbZu3UqpVGJqaorz589TKpWsfpv1yIM8+eSTvOtd7+JrX/vahhoOgMcff5w9e/awa9cuAF73utdx3333rcl4jI2N8b3vfY/Xv/71aJpmlcRNisLFGGx1KfCsNh7L4ZD8LENR/v/27jYkqjYN4Pj/Hkv3g5VhZqOTfUiMyMTksWyXjLa3xSwpKLNl3UqRCZKkFx7MoBfIgsglWEly7RXJsLSozHqUkggNg4ZWHSvcLJWsxNIyNdR7P6SDPdWTTuOcmfH+wXzwzMyZ68hwzX1f55z70hEREUFERAR9fX2YTCby8/PJyMhg6tSpxMTEEBUVZbm8vK2tDbPZTGho6FdXfbq7u+Pv74+/v7/luoy6ujo6OzuZNGkSkydPtmq1c5PJRHJyMoWFhQQEBNjs2L+nqanpi1PdBoOB+/fvW7WvgRFGXl4eLS0tAJbEcebMGe7duwfAzp07CQoK+snIHY9TJ4+SkhKtQ3AaOp2OsLAwwsLCSE9Pp6qqivz8fFauXImPjw9z5szBZDJx7ty5H14uPnbsWPR6PXq93tKWor6+no6ODssaH0NpS1FVVYXRaOTixYuWkcBI+2ZjKyunGAPv0+v1lJeXW7ZXV1dTU1NDRkYGFy5csFux2d7st9ij4jCEEMyePZsDBw5QWVlJfHw8p0+fpr29nfXr15OTk2Pp4/sjbm5u+Pr6EhISwty5c5k4cSKNjY1UVFRQW1tLa2vrN/djNptJTEwkLy/Prr/KBoOBhoYGy9+NjY34+fkNez8fP34kOzub9vZ2/Pz8+PDhg+U5b29v9uzZg6enJx4eHrx+/Zrc3FyamppscgyOwmWTR2FhIQaDgfLyclasWMHy5ct/an/FxcXMmDGDwMBADh8+bKMotSeEwGQycf/+fe7evcvx48fp6Ohgw4YNREdHk5WVxcuXL4ecSHx8fAgODmbevHn4+PjQ3NxMRUUFNTU1vHnzhu7ubp48ecKmTZvIzc21aaFyKMLDw3n69CnPnj3j06dP5OXlsWrVqmHv586dO5SVlbFw4UKuX79Oa2sr79+/Bz7XOAaKyq2traSkpNDV1TXi9Rx7c/mzLbZgjwq9o5FS8vz5c8uaJFJKq9ckkVJaml4lJibS09PDjh07MBqNdr1DdkBRUREpKSn09vayefNm0tLSrN5XSUkJHR0drF69moSEBMaPH8+ePXvQ6XRMmDCBs2fPUldXx/79+wGrzsQ47NkWlTyGoLy8nH379nHz5k0ADh06BEBqaqqWYdnNwJokly5doqCggK6uLqKjo4mJibHc2TsUL168IDY2lm3btmE2m2lra+PEiRMjHP3IGHxWpa+vj0WLFhEXF0dtbS1VVVVMmTKFzMxMuru7LRe7DX7PMDhs8nDqgqm92LJC74yEEPj5+ZGcnMzWrVsta5Js376dd+/eERUVRUxMzB+uSdLU1ERcXBxZWVnMnz/fzkdge4OTgE6nY86cOUybNg2j0YjJZMLT0/OLRZKklHbtJ2MPrnU0I8SWFXpnJ4TA19cXo9HIrVu3LK0o0tLSiIyMJD09nerqavoGtb1sbm4mNjaWY8eOuUTiGGzgu9HT00NlZSUAoaGhBAYGfvE6V/y+qOQxBLaq0Lsib29vEhISKCoqorS0lKCgIA4ePMiCBQvYu3cvt2/fZu3atRw5coTIyEi7xZWfn8+sWbPQ6XQ8ePBgxD8vPDzc5UYWPzK6jtZKtqrQuzovLy/i4+O5fPkyZWVlhIWFsXv3bhITE1m8eLFdYwkODqagoGDEE9bAiMLDw2PIzbxdhSqYDpEtK/SK/bjAvU0OO99RBdMhioqKsmX/0S84SHMhRRkWNW1xABs3bqS4uFjrMJzOkiVLCA4O/upx5coVrUMbFdTIwwFERkZSX1+vdRhOR93bpC018lAUxSoqeSguydb3NilfU2dbHIQD9AdRHJPDnm1RIw9FUayiksd3dHZ2cuPGjW+uVGZrcXFxzJ8/n8ePH2MwGMjJyRnxz1SUn6WmLd/Q29uLm5sbW7ZsISAggNTUVLq6unj16tV32zw6soaGBuLj42lubkan05GUlMS2bdu0DmvYdu3axdWrV3F3d2f69OmcOnUKLy8vrcMaaWra4kwGOrG9ffsWg8EAfL79PikpiYKCAi1Ds8qYMWM4evQoZrOZiooKMjMzqamp0TqsYVu6dClVVVU8evSIoKAgy9IIijZU8vgOKSXr1q2jtLSUkJAQent7OX/+PGvWrNE6tGHT6/WEhYUBMG7cOGbOnOmUS+ItW7bM0hkvIiKCxsZGjSMa3dRFYoMMXuWppaWFa9eucfPmTbKzs4mOjrbcfm3loi4Oob6+nocPHzJv3jytQ/kpJ0+eJDY2VuswRrWfqXm4LCHE34AkIACoAFKklD3aRvXzhBCeQBlwUErpkPMvIUQJ8K1GJ2lSyiv9r0kDfgHWSPUF1owaeQwihJgBrAH+DvwK/Bf4D/An4IMQwguIAxqAIill33d25XCEEGOBS0CuoyYOACnlkj96XgjxTyAaWKwSh7acc+w9cn4B/Pn8xbwO6AFPKeUHIYROSvkOqAW240SJV3yei+UAZillhtbxWKt/RPgrsEpK+VHreEY7lTwGkVLmSim3Silf9W96A9QLIf48aJTRCZiklJ+EEM7y//sL8A/gr0IIU/9jZNYXGFn/BsYBv/UfQ5bWAY1mquYxiBDCTUrZ+7ttAUB7/6gDIcRuoFJK+Vv/aMRppi6KYkvO8stpF79PHP3bXkgp3wkhfIUQ/wI2Av/rf04lDmXUcpp5u9aklK+EENlAMxAK1GkbkaJoS01bFEWxipq2DIMQQidcsQGHolhBjTwURbGKGnkoimIVlTwURbGKSh6KolhFJQ9FUayikoeiKFZRyUNRFKuo5KEoilX+D/8caI8ilQs+AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "visualize_fun(linear_module.weight.t(), 'Dataset with learned $w$ (PyTorch SGD)')" ] }, { "cell_type": "markdown", "id": "ordered-expression", "metadata": { "id": "iw41EqhA4sqZ" }, "source": [ "# Neural Network Basics in PyTorch\n", "\n", "Let's consider the dataset from hw3. We will try and fit a simple neural network to the data." ] }, { "cell_type": "code", "execution_count": 14, "id": "tender-quick", "metadata": { "id": "O3aNIZxJ4sqZ" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEYCAYAAABRB/GsAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAnc0lEQVR4nO3df5hcVZkn8O/bTQU64NJhaFco0iYyGjA2pMceEzfrjjBI+CGxBDXGoDszLnl8ZnTBZVoT7DGJEyVjdoVn1p2ZJzrOjJMIAYllQnACLKAj0tGO1UnTQlz8QYfCXeOQRiWtVLrf/aOqkurqe2/d6rr3nnPv/X6eJ8+TvnW7+9zu6vvec8573iOqCiIiSq820w0gIiKzGAiIiFKOgYCIKOUYCIiIUo6BgIgo5RgIiIhSjoGAiCjlGAiIiFKOgYBSR0R+KiJXRPS9FolIQUR+JSL/1eWcLhF5SESOicjfi8jtInKLz6//XRFZHGijKXVOM90AIpuJyE8B/BdVfXiWX+JjAB5T1V6Pc9YD+D+q+jYR6QIwDOB3fX79/w7gUwBumGX7iNgjIArZqwGMNjjnCgD3Vv7/RwAeUNUJn19/N4DLROS82TWPiIGAEqoy/LNeRH5QGXL5BxE5w+G8i0XkMREZF5FREVlZ89o/A+gGsEdEfi0iH2vy8x8BcBmAz1c+/3V1nztHRF4E0FP5HiMArgbwzbrzPisiX6v5eKuI/G8RyajqbwAcAHDl7H5SRAwElGxrAKwAcCGA1wEYqH1RRDIA9gB4EMArAXwEwA4RWQQAqvp+AGMArlPVs1T1s01+/uUA/hXAhyuf/8Paz1fVlwG8GcDPK6/3oBwUDtddx1+h/NS/REQ+BOAqANeraqny+lMALp3ND4gIYCCgZPu8qh5R1RcAfBrA6rrXlwE4C8AWVX1ZVR8BcL/DeW5a/XwAWALgYM3HnQB+VXuCqv4bgDsBfBnl+YRrVPXFmlN+Vfk8ollhIKAkO1Lz/2cBnF/3+vkAjqjqVN15WZ9fv9XPB2YGgmMAXuFwXgHl3sJ6VT1S99orAIw38T2JpmEgoCSbX/P/bgDP173+PID5ItJWd16x5mOvDTv8fH4jl2J6IDiE8jDWSSLSA+BvAfwTgD9x+BoX130NoqYwEFCS/ZmIXCAi5wC4DcDOutf3A3gJwMdEJCMibwVwHYC7a875fwBe4/L1/Xx+I/WB4AEAf1D9QESyKM9DfAjAnwLoqXyf6uunA3gjgIea+J5E0zAQUJJ9BeWJ3B9X/m2ufbEyWbsS5UydXwD4GwAfUNWna067HcBAJSvoz2fx+a5E5FUA5gGoPf/LAK4RkQ4R+XcoB4bPqepuVT0OYCvK8x1VK1Fep1Df2yHyTbhVJSVRAAvBjBGRz6CcSXSnj3P3A/igqj4ZesMosbiymMgyqnpbE+cuDbMtlA4cGiIiSjkODRERpRx7BEREKRfLOYJzzz1XFyxYYLoZRESxcuDAgV+oalf98VgGggULFmBoaMh0M4iIYkVEnnU6zqEhIqKUYyAgIko5awKBiLRXtvS733RbiIjSxJpAAOBmlOuqExFRhKwIBCJyAYBrAXzRdFuIiNLGlqyhO1He5NupDjsAQETWAlgLAN3d3dG0isgiA/kR3LX/CCZV0S6C1UvnY3Oux3SzKAGM9whE5O0oF9g64HWeqm5T1T5V7evqmpEGS5RoA/kRbB8cw2SlEsCkKrYPjmHNF54w3DJKAuOBAMByACsr1SLvBnC5iGw32yQiO+QLRSzf8gi2D445vv74j15AvtDMPjhEMxkPBKq6XlUvUNUFAN4L4BFVvdFws4iMyxeKWL9rBMXxCc/ztu6r3+ueqDnGAwEROdu0ZxQTpcmG5z3fIFAQNWLLZDEAQFUfA/CY4WYQGTeQH8Gx4yVf557f2RFyayjp2CMgsky+UMQOlzmBeh2ZdvSvWBRyiyjpGAiILLN132F47RLS2ZGBAMh2duD263uQ681G1TRKKKuGhojIe8y/syOD4Q1XRtgaSgP2CIgs4zbmLwA2rlwcbWMoFRgIiCzTv2IROjLt044JgDXLuh2HgaprDRau24vlWx7hugJqGoeGiCxTvdlv3XcYz49P4PzODvSvWOQaBNbvGjmZZlocn8D6XSPTvg5RIwwERBbK9WZ93ci37js8Y63BRGkSW/cdZiAg3zg0RBRjbhPLjVYjE9ViICCKMa+JZc4VkF8MBESWmM2kb/+KRRCH4wrWICL/GAiILFBbYE5xatK3UTDI9WZdF5+xBhH5xUBAZAGvSd9Gsi7DQ51zM4G0jZKPgYDIAm5P736e6vtXLEKmfeYA0a9/c4LzBOQLAwGRBdwmff1UFs31ZnHmnJmZ4KUp5TwB+cJAQGQBp9XEzVQWfXHCuWQ15wnIDwYCIgvkerO4/foeZDs7ZlVZ1K3ncHYH5wmoMa4sJrKE39XETvpXLEL/vQdRmpqeQ/TL35SQLxS5ypg8Ge8RiMgZIvJdETkoIqMissl0m4jiJtebdZwwnlJg4+5RAy2iOLGhR/BbAJer6q9FJAPg2yLyDVUdNN0wojg5XppyPD7uMn9AVGU8EKiqAvh15cNM5Z/XBk1EiZEvFH1VGSUKk/GhIQAQkXYRGQbwcwAPqep+h3PWisiQiAwdPXo08jYSBW22q4ndzPNYQMb1BOTFikCgqpOqugTABQDeJCJvcDhnm6r2qWpfV1dX5G0kClorq4mdbLjOffcyricgL1YEgipVHQfwGICrzLaEKHytrCZ24jWkxLLU5MV4IBCRLhHprPy/A8AVAJ422iiiCLSymthNuzjVInU/TgRYEAgAnAfgURE5BOB7KM8R3G+4TUSha3U1sZNJdc6zcDtOBNiRNXQIQK/pdhBFrZm9if3KdnY4DgO5VSglAiwIBERp1spqYif9KxZN28weaL2XQcnHQECUIGH0Mij5GAiIEiboXgYlnw2TxUREZBADARFRyjEQEBGlHAMBEVHKcbKYKKFY2ZT8YiAgSqBqZdPqeoJqZVPAuyYRpROHhogSKOjKppRsDARECRR0ZVNKNgYCogQKo7IpJRfnCIgSKM01h/KFIjbtGcWx4+W9mjs7Mti4cjHnRjwwEBAlUJpqDg3kR3DX/iOYVIUAEAGmaqpuj0+U0H/vQQCcKHfDQECUUGmoOTSQH8H2wbGTHysAp60XSlOKrfsOJ/7nMVucIyCi2Lpr/xHf5xbHJ5AvFENsTXwZDwQiMl9EHhWRp0RkVERuNt0mIoqHZndeW79rhMHAgfFAAOAEgFtV9WIAywD8mYi83nCbiCgGmt2LmWspnBkPBKr6M1X9fuX/vwLwFAAO5BFRQ6uXznc8fvpp7rc2p6080854IKglIgtQ3r94v8Nra0VkSESGjh49GnnbiMg+m3M9uHFZ98meQbsIblzWjcObr/bcp5nDQ9OJNjnGFhYROQvANwF8WlV3eZ3b19enQ0ND0TSMiGIpXyjilp3Djq91dmQwvOHKaBtkARE5oKp99cet6BGISAbAfQB2NAoCRER+eKWKjk+UImyJ/YwHAhERAH8P4ClV/Zzp9hARpY0NC8qWA3g/gBERGa4cu01VHzDXJDvlC0Vs3D0642lGpLyIJpvg1aPUmiTtTdDMtcybmzlZaqLWmXPaw25mrBgPBKr6bQDN5YClTH3tlHrVaZ7amvNDz76AHfvHTr42N9OGz1x/SWz/+Gn2krQ3QbPXsuG6xbj13oOYnJo+F/ryiSnkC8XYXX9YjA8NkbfqG98tCNSbKE1i/a5D2D44Nm2p/fHSFG7ZOYzXrN+LgfxISK0lGyVpb4JmryXXm8UrTp/5vFstOUFlDASWc3rjNzJRmnJ9bUqB7YNjWPOFJ1ptGsVEkvYmmM21vOgyMRzH6w8LA4HlwnqzPv6jF9D7qQeZT50CSdqbYK7L2H7n3Izr5yTp+sPCQGCJfKGI5VsewcJ1e7F8yyMnb9DNvlk7Mu1o8znjcux4ibVXUqB/xSJ0ZKbfQOO4N0G+UMRLLzv3jr2WQyXl+sPEQGCBgfwIPrpzGMXxCShOTYDlC0XHN3G9armVbGcHbr++B+9b2u37e0+UJrFx92gLrSfb5XqzuP36HmQ7OyA49T6J20Sp15i+2/APkJzrD5M1K4ubkZSVxQP5kWmZPfWynR14fN3ls0r9a/S16925agn/MEKSpNRNkxau2wu3t3P1b6WRtP8u3FYWG08fTav6DTWcVOcHZrPByOZcDzbnegAAa77wBB7/0Que53PTjnAkKXXTtPM7OxwLxgnga5iHvwt3HBoyxM+GGkFNZu246c24c9USdHa4T6gxgyIcSUrdNM1pmFQArFnW7etGzt+FO/YIIlTbLW00YuP3Kcevaq+i91MPOq5JaBPBwnV7U9ldDlOSUjdNa3UfZv4u3DEQRGQgP4Idg2MNAwDQ3FNOszZct3ha97iqutNTcXwCH905jKFnXzg5tESz5zacwdTF2WllH2b+LtxxaCgC1fkAP0FgbqYNd6xaEtpNuD6DwmmHJwWwY3CMaaUBYOqiPfi7cMceQcjyhSJ2NJgUBso35NVL50fyFF77VLVw3V7HcxTArfccPHk+zU6rwxkUHP4u3DEQhMhPZpDftLewuHWXgfJwEbMqWpP2dMWgBPVzbGVoKck4NBQSP0Eg6Anh2ehfsciz9CuzKmavmq7otFCQ/MsXiui/9+C0n2P/vQf5cwwQA0EI/AQBILwJ4WbkerNYs6zbMxhws+/ZYbpiMDbuHkWprox0aUq5Ij5ADAQBW/OFJ3z1BG5c1m1NVs7mXA/uWLXEceK4iqWrm8d0xWC4bSvJ7SaDwzmCAOULxYYreAHgDgvLOVTb89Gdw47ZTdsHx9D36nOsa7fNmK5ot4H8CO7afwSTqpEma9jIih6BiHxJRH4uIk+abstsVCuH3rJzuOG5N1owHOQm15v1THHdtIdd8WZcdlFXU8fJ2TyXEtNux/2oDt9W189MqmL74Fhqe75WBAIA/wjgKtONmI3aCcFGbBoOcpP1eFo9drzECbomPPr00aaOR8Wt5Lmtrr3kvBnHMu2CDdctnvXXdCvx4qf0SxJZEQhU9VsAGo+pWGjTnlFfO4jFIQgAjbOYmK3hn41zBHHLZMoXirjvwPS2CYBVvz+/pZ71pEtZXrfjSWdFIPBDRNaKyJCIDB09avaJCii/Qd3q9tRbfuE5sQgCQHl4qCPj/rZgtoZ/Nu6MFbdMJqf2Ksz3qpImNoFAVbepap+q9nV1mR1j9buhfLazA3euWoIdN705opYF4/brL0HGY5szZmv4Y2NJAxt7KV7i1t64ik0gsImfDeXvXLUEj6+73NqJYS+53iy2vvtSz3NsHUqwiY07Y9nYS/Hithdxq+31SpVO43ubgWAWGj2NdHZkYhkAauV6s55ZGbYOJdgm15vF4+sux0+2XGvFg4GNvRQ3+UIRLzr0ujPt0nJ7Vy+d7/paGt/bVgQCEbkLwBMAFonIcyLyQdNt8uL1NNKRacfGlbPPZrCJV1YGu+bxVO2l1Ab500+z4jYww8bdo5hyOH5am7QcUL3m7NL43rZiQZmqrjbdhkZqi151zs0g0yYzlr13dmSwceVi4099Qcn1ZrFx96jjnICtQwnkz29Kp26x4xMlK4sLus1FTZScwkPzslzwd5KdjwIWyReKWLLpQdyyc/hkyt2x4yVAyjf+6tjvnauWYHjDlVb9IQVh48rFM4YSAOCl355I5ViqXzbn6sctcygscRomC5sVPQJbVase1j/5A0BpUnHm6adheMOVBloWnWpg27RndFqWlK1PkTawfZP0uGTitAng8KcHj4S2pnB/glPYI/DQf++wYxCosu0PJyy53izmzpn5zJDGp0g/bH/ijkvmkNufnsefZNNyvVn0r1iE8zs78Pz4BLbuO2xV7y0qDAQuBvIjaDQUadsfTpji8hRpA9t/VnEZEnHLWvMqg9KsuK20DgsDQZ3q2G6jUtI2/uGEKS5PkTaw/Wdl4/qGevlCEb/+zYkZx4NIHa1le+8tKpwjqOE1J1CrTWDdH07Y+lcsmjbuXfX8ixMYyI/EpoRGFJx+VrY9ONi+ZePWfYcd/w7PnHNaoO22vfcWFfYIaty261DDIAAAn3uPffsJhK36FDm3rg6RKlJdvtdJHJ64bed2I34x4PImtvfeosJAgFMF5I77yE9efmF6N2fJ9Wbx2xPOgXKHj60508S2FcVxE9UNOi7zJWFLfSAYyI/gozuHGxaQaxfBjcu6Y1dALmhuZXoV6azR4sTmNQRxEdUNmr23slTPEeQLRewYHPPclQsoZy8UPpns9QJ+tYu4BoOt+w6n7g+onu1rCOIiyhx/2+dLopDqQLB13+GGQQDwrrmTNquXznfNqPKzS1vSOW1UVM1CSfvNplm8QUcn1UNDjTIDBHbvMWzC5lwPzpwzs+QEUP55pXkYJF8oug4xpi0LheIl1YHAa+KpsyODO1YtYVqkg0+/swdOq/wV6SzhW+W1c5utWSicz7BT1L+X1A0N1VYRPbsjg0y7oDR5aoBIAKyJyf7CpuR6s7hl57Dja2l+8vXauc3GLBTOZ9jJ7fcy9OwLePTpo6HMmaQqENT/gMcnSsi0CebNzWD8eCnVRaea5VbCt00E+UKRP8M6Nv48vFbV2tjepKs+pDr9XU2UJqcltgQdtFMTCPKFIm695+CMjJfSlGLunNOYFdQkt5XGk6qpfaoMu1pm0GxdVVvba0/Lw9lAfqRhWZv6t1aQQTsVcwTVnoBb2qPpN34cVfOvnfZ+TWOtFiCaaplBsnFVbb5QRP9XD04rAtf/1YOJnrvIF4oNg4CboO5dVgQCEblKRA6LyDMisi7or99os3lbJ/Jsl+vNYorB9SS3qphBVssMko2rajftGZ02ZweU9/7YtMd9Ij5oUU/U+nlocutUBnXvahgIRORhEbk0kO/m/PXbAfwvAFcDeD2A1SLy+iC/h9dNyfQbP+5sfKo0xcYbqxcbV9W6pd82WvkflKjLUucLxYbrb7KdHVizrDvU95afOYKPAbhDRJ4FcJuq/iyQ73zKmwA8o6o/BgARuRvAOwD8IKhvcL7LxGa7iPE3ftzFodJmVOK44xUXbU0X5QT6QH6kYY2ueXMzeHzd5QCAvlefE9p7q2EgUNXvA7hcRG4A8C8isgvAZ1U1qL5/FsCRmo+fA7C0/iQRWQtgLQB0d3c39Q3cblYMAq2L480vTLyxtkYwc1K0ejwKUU2g+ylv094m06oahPne8pU1JCIC4DCAvwWwGcBNIrJeVf85gDa4rU2afkB1G4BtANDX19fU9BtvVuHizS/ebMrScfvDjmq+vXNuxnEYqtNlt7TZalTeZt7cDDZctziy30PDQCAi3wbwGgCjAAYB/BGApwHcLCJvUdW1LbbhOQDzaz6+AMDzLX7NGXizIprJtkVlbutToppwd8l9cD0+W17zAtnOjpPDQVHxkzX0IQBZVX2bqv6Fqt6vqs+o6kcAvCWANnwPwGtFZKGIzAHwXgC7A/i6RNSAbVs1XnZR14whgijnnNw2vglyQ5x8oeg61CUwswq9YSBQ1SdVXePhta02QFVPAPgwgH0AngJwj6pGlytGlGI2LSrLF4q470Bx2pCJALjhjdH15qPIgvMaFlpjqMhlS+sIqpk+rVLVB1T1dap6oap+OoivSeawkFl8uN3gzu4IdkzcD6feiQJ49OmjkbXBKQUYAI6/fCKw97FXkDVV48yKBWWUHFHnYdsirsGvf8UiZBxqYLwU4I3PLxt6J9W1FZ11gfDY8VJg72O34Gty4SEDAQXKtjHnKMQ5+OV6szjrjJk5I6VJjfx3ZsvixFxvFmeePvNnEtT72MaFhwwEFCgbnuqiFvfgN27JZjo23SDDfB/buKI7NdVHKRpuq7iTXHIi7sHv7I6M414KUc8T2LTeJ4z3sU3rNeoxEFCgnFZxC8ppgUkV9+DnUEDW83iYbFnvE3TpFNvWa9Tj0BAFKtebxQ1vzE7Lk1YA9x0oxmLMfDZsGtKYDbehIbfjYbFpwj3o4Rvbhw/ZI6DAPfr0UcdNNDbtGbXi6SdoNg1pzIYNPZrqPgTVEtTVfQgAc0/MQfZO3FYS2zJ8yB4BBc7tzX3seCmRvQKbx379sKFHY8M+BG4G8iO4cP0DWLBuLy5c/wAG8iNNfb7XSmJbhg8ZCChwXm/ujbvN/2EHKc6po1U2ZLGY3ofATXULyeruhpOq2D441lQw2LRn1LWiqi3DhxwaosD1r1iEW3YOO77mlJ0SZ0nZAN6WSVrb3LX/iOtxP6uA84WiazBT2DFRDLBHQCGw5c0dhbinjtqifiVvo+NRcdvn3O14Pa/JYJu2MGUgoFDMc6nf7nY8rmxZDRt3b7/0vBnHMm2CjSsXO5wdnXaXHFq34/W8HghsGRYCGAgoJBuuW4xM+/Q/lkz79B2XksCGida4q1YdrSUAVr1pvvHe5eql85s6XitfKKLNJWB0dmSMX1stzhFQKOKeUulXWq4zTDZUHXVTnQe4a/8RTKqiXQSrl85vOD9QTSJwGkLqyLQb7+nUE/etBuzV19enQ0NDpptBlDgmUmEXrtvrmlXzky0tb3lixMV/8Q1MlKZmHG8Xwf94z6XGHhRE5ICq9tUfZ4+AIhH3XPs0MFUGwYYFbUEayI84BgEAmFK18n1vdI5ARN4tIqMiMiUiM6IUJUMScu3TwFQZhDjNs/gpg/GV/WOun29rcDM9WfwkgOsBfMtwOyhEttdZoTJTqbA2LGjzw88DTb5QxJTHaLuNwQ0wPDSkqk8BgJgoc0iRcauz4naczDA5RBOHBW1+Fg96lcRoE3vX2JjuEfgmImtFZEhEho4eNZ9NQP61motN0YjTEI0JXj2m6pCRV0mM9y3tDqtpLQu9RyAiDwN4lcNLn1DVr/v9Oqq6DcA2oJw1FFDzKAKtrs6kaDAV1ptbj0kB15IqtUxtTO9H6IFAVa8I+3uQ3bIuf0A2LbGnsjgM0ZjitFmNX6ZLZTQSm6Ehii8OOZAbmzajaaR+Utvv0KYNpTIaMTpZLCLvBPA/AXQB2Csiw6q6wmSbKHgcciAntm/f6KS2x7Rw3d6G52dj8l43nTX0NQBfM9kGigaHHKhe3Et4u80ZVN25akksrgPg0BAZEKfhAApP3Et4969YhDaX0aHlF54TmyAAMBBQxLjKmKriXsI715vF596zBHMzp26jIsCNy7qx46Y3G2xZ81hriCIV9+EACo5TFk7ckgiSMuTJQECRivtwAAWHSQT2YCCgSCWt0iS1JilP1HHHOQKKFNcUxAMn9NOFPQKKVPXpb+PuUYxPlOuynJHh84hN4pjfT63hXyAZ8dsTpzbuOHa8hI/uHMZAfsRgi/xJw5Myy4anDwMBRc5tj9odg2NW31jTkvrKCf30YSCgyLndUBSw+qkzLU/Kcc/vp+YxEFDkvG4oNm9Wk5Yn5Sgm9NMwxBYnDAQUOa8bis2b1aTlSTnsrSPTMsQWJ8waosjlerOuG3nYvFnNgt9xXgNx2UVdBloTrjDz+7m63D7sEZARbpvS2LpZTb5QxHd+9ILja48+za1Tm8E9rO3DQEBGxG1h2dZ9h+HWV0naHEGtMMbyuYe1fTg0REbErc6M180+aXMEVWEtLOMe1vZhICBj4lRnxq1GksB78jvOwhrL5x7W9jE6NCQiW0XkaRE5JCJfE5FOk+0hcnPZRV2oH7gQAGuWdccmmDUrrHTZuA0LpoHpOYKHALxBVS8B8EMA6w23h2iGfKGI+w4Up80RVIPA5lyPqWaFLqx02bDTU6l5pvcsfrDmw0EA7zLVFjIrXyhaO1/gVhIj6dlCYW4cE6dhwTSwaY7gTwDsdHtRRNYCWAsA3d3dUbWJImB7tcu0rCiuF7cJfZq90AOBiDwM4FUOL31CVb9eOecTAE4A2OH2dVR1G4BtANDX18f0ggRxm5TctGfUiptOmjfT4ZN7OoQ+R6CqV6jqGxz+VYPAfwbwdgBrVJk/lkZuT9bHjpesKDvAyU1KOtNZQ1cB+DiAlap63GRbyByvJ+tNe0YjbIkzTm5S0pmeI/g8gNMBPCTlVYWDqvohs02iqPWvWORae+jY8VK0jXHBIZLW2ZwQkHams4Z+1+T3Jzt4FaGjZLA9ISDtTK8jIAIAdHZkmjpO8ZKWTX3iioGArLBx5WJk2qav3c20CTauXGyoRRSktKbgxgUDAVkh15vF1ndferLeTLsISlOKrfsOW5E5RK1Jy6Y+ccVAQNbI9WZPpmpWK1Fy96pkYAqu3UxnDRFNY9vuVcx0CQZXKduNgYCsYtNYMjNdgsUUXHtxaIisYtNYMjNdghPGTmcUHAYCsopNY8k29U7irNqzKo5PQMF5HxsxEJBVbCrnYFPvxBazebJnz8p+nCMg69gylhxmPf44mu2cCXtW9mOPgMiFTb0TG8z2yZ49K/uxR0DkwZbeiQ1m+2TPnpX9GAjIaqby+Ll+YCa3DXrOyHgPLHANgf0kjnvB9PX16dDQkOlmUMjqx6SB8pNk2MMzpr6v7fKFIv7bzmFMObx247JubM71RN4mao6IHFDVvvrjnCMga5nKNmGWi7NcbxYQ59fu2n8k2sZQoBgIyFqmsk2Y5eJuymUAYTKGIwt0iumtKv9SRA6JyLCIPCgi55tsD9nFLaukTSTUxUjMcnHXLs5dArfjFA+mewRbVfUSVV0C4H4AnzTcHrKI0ypjoPz0GebKVJtWN9tm9dL5jsfnnBZucKZwmd6q8pc1H54JgP1LOqk6MXvrPQdnDD2EVZG0mi00UZpEuwgmVZFllstJ1Qnhr+wfmzZMNFGamra4bCA/grv2H8GkKtpFsHrpfE4mW8x0jwAi8mkROQJgDTx6BCKyVkSGRGTo6NGj0TWQjMr1Zl3Hn51SGVuRLxTR/9WDJ7/upCoy7cIgUGdzrgfnnT1zmKwanAfyI9g+OHby9zapiu2DYxjIj0TdVPIp9EAgIg+LyJMO/94BAKr6CVWdD2AHgA+7fR1V3aaqfara19XVFXazySJe489BDkds2jOK0uT0oFOaVGzaMxrY90gKt4nz4vgEtg+OOb7GzCJ7hR4IVPUKVX2Dw7+v1536FQA3hN0eih+vjJQgUzqPHS81dTzNZjNxzswie5nOGnptzYcrATxtqi1kr6zHTYcpnWa4TeR7YWaRvUzPEWypDBMdAnAlgJsNt4cs1L9ikds6psBSOr2GmDo7MoF8jySpFuRr5ubulnFE5hkNBKp6Q2WY6BJVvU5VmX9GM+R6s1izrHtGMAgypdNriGnjysWBfI+kyfVmMeVzuGdupo1ZQxYz3SMg8mVzrgd3rFoSWkloryEmZgy589Mjy7QJPnP9JRG0hmaL1UcpNsIsCe1WWdNrfoLKw3b99x5EyaX2RGdHBhtXLmYwtRwDARFYM3+2qjf4jbtHMT5Rzq6aNzeDDdfx5h8nDAREYM38VnDznvhjICCq4A2N0oqTxUREKcceAcUai5sRtY6BgGKrWtysqlrcDACDAVETODREseVWxIzFzYiaw0BAseVWxIzFzYiaw0BAseVV52b5lkcc6wflC0Us3/IIFq7b63oOUdowEFBseRUxK45PzNjOMl8oYv2uERTHJ6Au5xClEQMBxdbmXA9uXNbt2jOYKE3i1nsOnrzRV7egrD8nyD0NiOJINIbjqX19fTo0NGS6GWSRhev2em54Padd8PKk8xkC4Cdbrg2lXUQ2EZEDqtpXf5w9AkqERlUw3YKAn88lSjoGAkoEvztmhbmnAVFcMRBQIvjdMUuB0PY0IIorK1YWi8ifA9gKoEtVf2G6PRRP1Rv6LTuHXc/Jdnbg8XWXR9Qiongw3iMQkfkA3gZgrNG5RI3kerNYfuE5jq+1CTgMROTAeCAAcAeAjwGeSR9Evu246c24cVk3akeJ5mba8Ln3LOEwEJEDo+mjIrISwB+q6s0i8lMAfW5DQyKyFsBaAOju7n7js88+G11DiYgSwC19NPQ5AhF5GMCrHF76BIDbAFzp5+uo6jYA24DyOoLAGkhElHKhBwJVvcLpuIj0AFgI4KCU+/AXAPi+iLxJVf9v2O0iIqIyY1lDqjoC4JXVjxsNDRERUThsmCwmIiKDrFhHAACqusB0G4iI0iiWRedE5CiAZtOGzgWQxmEnXne68LrTpdnrfrWqdtUfjGUgmA0RGXJKm0o6Xne68LrTJajr5hwBEVHKMRAQEaVcmgLBNtMNMITXnS687nQJ5LpTM0dARETO0tQjICIiBwwEREQpl7hAICJXichhEXlGRNY5vC4i8teV1w+JyO+ZaGfQfFz3msr1HhKR74jIpSbaGbRG111z3u+LyKSIvCvK9oXFz3WLyFtFZFhERkXkm1G3MQw+3udni8geETlYue4/NtHOIInIl0Tk5yLypMvrrd/TVDUx/wC0A/gRgNcAmAPgIIDX151zDYBvoLx97TIA+023O6Lr/g8A5lX+f3VarrvmvEcAPADgXabbHdHvuxPADwB0Vz5+pel2R3TdtwH4q8r/uwC8AGCO6ba3eN3/CcDvAXjS5fWW72lJ6xG8CcAzqvpjVX0ZwN0A3lF3zjsAfFnLBgF0ish5UTc0YA2vW1W/o6rHKh8OolztNe78/L4B4CMA7gPw8ygbFyI/1/0+ALtUdQwAVDUJ1+7nuhXAK6Rc0vgslAPBiWibGSxV/RbK1+Gm5Xta0gJBFsCRmo+fqxxr9py4afaaPojyE0TcNbxuEckCeCeAv4uwXWHz8/t+HYB5IvKYiBwQkQ9E1rrw+LnuzwO4GMDzAEYA3KyqU9E0z5iW72nWFJ0LiDgcq8+P9XNO3Pi+JhG5DOVA8B9DbVE0/Fz3nQA+rqqTIk6nx5Kf6z4NwBsB/CGADgBPiMigqv4w7MaFyM91rwAwDOByABcCeEhE/lVVfxly20xq+Z6WtEDwHID5NR9fgPKTQbPnxI2vaxKRSwB8EcDVqvpvEbUtTH6uuw/A3ZUgcC6Aa0TkhKrmI2lhOPy+z3+hqi8BeElEvgXgUgBxDgR+rvuPAWzR8uD5MyLyEwAXAfhuNE00ouV7WtKGhr4H4LUislBE5gB4L4DddefsBvCBykz7MgAvqurPom5owBpet4h0A9gF4P0xfyqs1fC6VXWhqi7QcpnzrwL405gHAcDf+/zrAN4iIqeJyFwASwE8FXE7g+bnusdQ7gVBRP49gEUAfhxpK6PX8j0tUT0CVT0hIh8GsA/lDIMvqeqoiHyo8vrfoZw5cg2AZwAcR/kJItZ8XvcnAfwOgL+pPB2f0JhXa/R53Ynj57pV9SkR+RcAhwBMAfiiqjqmH8aFz9/3XwL4RxEZQXnI5OMa810PReQuAG8FcK6IPAdgA4AMENw9jSUmiIhSLmlDQ0RE1CQGAiKilGMgICJKOQYCIqKUYyAgIko5BgIiopRjICAiSjkGAqIAiMijIvK2yv83i8hfm24TkV+JWllMZNAGAJ8SkVcC6AWw0nB7iHzjymKigFR2ATsLwFtV9Vem20PkF4eGiAIgIj0AzgPwWwYBihsGAqIWVXaD2oHyTlEvicgKw00iagoDAVELKiWedwG4VVWfQrn65UajjSJqEucIiIhSjj0CIqKUYyAgIko5BgIiopRjICAiSjkGAiKilGMgICJKOQYCIqKU+/8BZJ2P5/PYwQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "%matplotlib inline\n", "\n", "d = 1\n", "n = 200\n", "X = torch.rand(n,d)\n", "y = 4 * torch.sin(np.pi * X) * torch.cos(6*np.pi*X**2)\n", "\n", "plt.scatter(X.numpy(), y.numpy())\n", "plt.title('plot of $f(x)$')\n", "plt.xlabel('$x$')\n", "plt.ylabel('$y$')\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "forward-soldier", "metadata": { "id": "jix59ZfA4sqa" }, "source": [ "Here we define a simple two hidden layer neural network with Tanh activations. There are a few hyper parameters to play with to get a feel for how they change the results." ] }, { "cell_type": "code", "execution_count": 15, "id": "instrumental-enhancement", "metadata": { "id": "K947YLD74sqa" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "iter,\tloss\n", "0,\t3.49\n", "600,\t3.40\n", "1200,\t2.95\n", "1800,\t1.60\n", "2400,\t1.37\n", "3000,\t0.82\n", "3600,\t0.70\n", "4200,\t0.48\n", "4800,\t0.32\n", "5400,\t0.24\n" ] } ], "source": [ "# feel free to play with these parameters\n", "\n", "step_size = 0.05\n", "n_epochs = 6000\n", "n_hidden_1 = 32\n", "n_hidden_2 = 32\n", "d_out = 1\n", "\n", "neural_network = nn.Sequential(\n", " nn.Linear(d, n_hidden_1), \n", " nn.Tanh(),\n", " nn.Linear(n_hidden_1, n_hidden_2),\n", " nn.Tanh(),\n", " nn.Linear(n_hidden_2, d_out)\n", " )\n", "\n", "loss_func = nn.MSELoss()\n", "\n", "optim = torch.optim.SGD(neural_network.parameters(), lr=step_size)\n", "print('iter,\\tloss')\n", "for i in range(n_epochs):\n", " y_hat = neural_network(X)\n", " loss = loss_func(y_hat, y)\n", " optim.zero_grad()\n", " loss.backward()\n", " optim.step()\n", " \n", " if i % (n_epochs // 10) == 0:\n", " print('{},\\t{:.2f}'.format(i, loss.item()))\n", "\n" ] }, { "cell_type": "code", "execution_count": 16, "id": "tutorial-slovak", "metadata": { "id": "ugpIyO484sqb" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEdCAYAAAABymAfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABJ3klEQVR4nO3deVxVdfrA8c+XRcUVVNwQXHA3VBT3yjJLszSymrSamabFmq3dyrKyvcZpaqbmN2XrTC5pZaRpmppaae6oaIq5goiKAq6ICN/fH4eLl8MFL3DvPedyn/frxUu5yznPUTjP/W7PV2mtEUIIEbiCrA5ACCGEtSQRCCFEgJNEIIQQAU4SgRBCBDhJBEIIEeAkEQghRICTRCCEEAFOEoEQAUIpNVIpNdLqOIT9KFlQJkTNp5RqCnxX/O3VWutjVsYj7EUSgRABQCn1b+ArIBgYrbX+s8UhCRuRRCCEEAFOxgiEECLASSIQQogAJ4lA2IpSap9SapiPztVZKZWslDqplHqgnNdEKqUWK6VylFIfKqVeVUo95Obx1yqluns06PLP9YlS6qWLvMYvrkX4XojVAQhRVUqpfcA9WuslVTzE48ByrXV8Ba+ZCPyqtb5aKRUJbAI6uHn8vwMvADdVMT5Pq0nXIjxIWgQikLUBtl3kNcOAz4v/fiewQGud5+bx5wJXKqVaVi08j6tJ1yI8SBKB8Lni7p+JSqlfirspPlZK1XHxuq5KqeVKqVyl1Dal1Gin5z4FYoB5SqlTSqnHK/n+74ErgXeK39/J9N5aSqnjQFzxOVKAa4EVptf9TSn1ldP3U5RSS5VSoVrrs8AG4Jpy/h2eVErtLu6a+kUpdaOLf6fHlFJblFLHlVKzHP9OSql4pdTG4vfOAsr8+1XmWiq6DoCLXYvwc1pr+ZIvn34B+4CtQDTQGFgJvOT03DAgFNgFPAXUAoYCJ4HOpuMMK+cc7rx/OUbXUnlxdgMOO32fBfQ1vaYJkAv0Au4HUoBGTs//C/hHOce/BWiF8YHsVuA00NJ0fWuLX9MY2F58jlrAfuDh4uu8GShw/BtW5Voudh0Xuxb58u8vaREIq7yjtU7XWmcDLwPjTM8PAOoDr2mtz2mtvwe+cfG68lT3/WDcFDc7fR+OkUxKaGOF7lvA/zD64EdqrY87veRk8fvK0Fp/rrU+qLUu0lrPAn4F+ple9q/i12QD84pjGoCRAN7SWhdorb8A1lXnWty4jgqvRfg3SQTCKulOf9+P8anXWSsgXWtdZHpdlJvHr+77oezNMwdo4OJ1yRjdLhO11umm5xpgfNIuQyn1O6XUpuKuq1zgEqCp6WWHnP5+BiO5tQIytNbOq0H3V3gl7l1LRdcBFVyL8G+SCIRVop3+HgMcND1/EIhWSgWZXpfh9H1Fy+Ldef/F9KT0zXMLYB5LiAP+A/wXuMvFMbqajuF4XxvgfeAvQBOtdThGd5lyI65MIEop5fzamIu8p8JrceM6oJxrEf5PEoGwyp+VUq2VUo0x+vFnmZ5fg9Fn/rhSKlQpdQUwCvjM6TWHgfblHN+d91+M+ea5ABji+EYpFYXRXXM/8Ccgrvg8judrA32AxS6OXQ8jkWUVv/YPGC0Cd/wMnAceUEqFKKXGULZLye1rudh1uHEtws9JIhBWmYFRDXNP8VepxVBa63PAaIzZLUeB/wN+p7Xe4fSyV4FJxV0rj1Xh/eVSSrUAIgDn1/8PGKmUClNKNcS4mf5Daz1Xa30GmIIx3uEwGmOdgrm1g9b6F+ANjJv6YYwumZXuxFZ8bWMwpoDmYAw0z6nitTRy4zoqvBbh/6TonPA5DywEs4xS6hXgiNb6LTdeuwa4W2u91euBVUFNuhZRPZIIhM/5cyIQoiaSriEhhAhw0iIQQogAJy0CIYQIcH5ZfbRp06a6bdu2VochhBB+ZcOGDUe11pHmx/0yEbRt25b169dbHYYQQvgVpZTLFejSNSSEEAFOEoEQQgQ42yQCpVRw8baB31gdixBCBBLbJALgQYx660IIIXzIFolAKdUauA74wOpYhBAi0Nhl1tBbGBuJu6r1DoBSajwwHiAm5mIVd4WoOZKSM3h+3jZyzhQAEB4WyuTR3UmMr8zWCkKUz/IWgVLqeozCVxsqep3WeqrWOkFrnRAZWWYarBA1UlJyBhO+2FySBABy8wp4aNYmJiWlWBiZqEksTwTAYGB0cSGyz4ChSqlp1oYkhPWSkjN4dPZmCgpdl4GZvjqNpOTK7LMjhGuWJwKt9UStdWutdVtgLPC91voOi8MSwlJJyRlMnJNCYQW1wDQwZVGq74ISNZZdxgiEEMUcLYGKkoDDwdw8H0QkajpbJQKt9XJgucVhCGGZpOQMJnzuXhIAaBUe5uWIRCCwvGtICHHB5LnbKChyLwmEhQYzYXhnL0ckAoGtWgRCBLrcvIJynwsNVtSrFcLxvAJahYcxYXhnmUIqPEISgRB+YsrNPeXGL7xCuoaEsJGIuqHlPi5JQHiLJAIhbOS5Ud0JDValHgsNVjw3qnuZ1369di9jnpxJuyfnM/i172VNgagySQRC2EhifBRTbu5JVHgYCogKD3PZJfTTu7OIG3k5s/92B/EHtpORm8fEOSmSDESV+OXm9QkJCVp2KBMBKTMTHn0UZs5kX3hLQorOU6SCGPmHtzlVuy5R4WGsfHKo1VEKm1JKbdBaJ5gflxaBEP6gsBDefhu6dIEvv+StweMYftc7PDDqcaJOZPHC4v8AkCELzEQVSCIQwu62boW+feGBB2DAANi6lc+vv4f80NpsbN2Vtwfdyphtyxj9y3IUSPeQqDRJBELYQFJyBoNf+77swK/WcMcdkJEBs2fDwoXQsSMThnfGMaT89qCxrI/qykuL/o+o44el/pCoNEkEQljMUWAuIzcPDaUHfpctg82b4ZVX4JZbQBm3/8T4KByje4VBwTx0/aMoNG/Oe4PD2acsuxbhnyQRCGGxKYtSySsoLPVYXkGh8cn+zTchMhJuv73M+6Kc6gwdCG/BM9f8ib4Zv/Do+i+8HrOoWSQRCGGx8iqI1t6zC775Bv74R6hTp8zzE4Z3LrXmIKn7lSR1G8K9y6ez4pO5XotX1DySCISwWHkVRP+cMh9q1YI//cnl84nxUdSrVbpKzDPX/IlDDZrS4ZH74ZR0EQn3SCIQwmIThncmLDS41GMtzp/hhk2L4bbboHnzct973FSk7mTtekwc8ReicjJhwQKvxCtqHkkEQlgsMT6KV8fElVpN/GFBMiFn8+Chhyp8r6vWxJroOAqCQmBDhduAC1FCqo8KYQOJ8VEXykgUFEC738HQodCzZ4XvmzC8MxM+31xqD4NzIaHsiGxD3YUriH3dm1GLmsLyFoFSqo5Saq1SarNSaptS6nmrYxLCUl98YawbePjhi740MT6K+nXKfp5LadGRpqlbSdp4wBsRihrG8kQA5ANDtdY9gV7ACKXUAGtDEsIiWhtTRjt1gpEj3XpL7pmym9mktOhAo/zTzJi53MMBiprI8q4hbVS9c0xvCC3+8r9KeEJUUlJyBlMWpXIwN+/CjmNn9sG6dfDvf0OQe5/TWoWHlakxlNKiAwDNdm71dNiiBrJDiwClVLBSahNwBFistV7j4jXjlVLrlVLrs7KyfB6jEJ5U3mrijOdegYgI+P3v3T6Wq32LdzZtQ35wCH2O7fFg1KKmskUi0FoXaq17Aa2BfkqpS1y8ZqrWOkFrnRAZGenzGIXwJFeriZtkZdDi+4UwfjzUq+f2sYz1BKWnn54LCSU1si1dM3Z6JF5Rs9kiEThorXOB5cAIayMRwrtcrSa+c8M8ipSCv/yl0sc7c66wzGNbm3ega+YuGTAWF2V5IlBKRSqlwov/HgYMA3ZYGpQQXlZm/r/WJP6ynB+6XwqtW1f/eMAWGTAWbrI8EQAtgWVKqS3AOowxgm8sjkkIrzKvJm6bc5CmZ47T+Ab3Zgq5Op7ZhQHjlKoFKQKGHWYNbQHirY5DCF9yLB5zzBoalrMbgPix11X5eM/P20aO01TSnZHGgPHA3H3VjlfUbJYnAiECVanVxPfNhfBw6Nq1ysd7blR3Js5JKRmELggOZWezdlyTJzuWiYrZoWtICLFqFQwc6PbaAVdc1SyqP6g/kTu3GgvVhCiHtAiEsFpuLmzbBrfeWu1DlWplALy/G76cBrt3Q4cO1T6+qJmkRSCE1VavNj6xDxrk+WMnJBh/SiVSUQFJBEJYbdUqo0uoXz/PH7t7d2NzG0kEogKSCISw2qpVRrnp+vU9f+xataBHD1i/3vPHFjWGJAIhrHT+PKxZ451uIYeEBNi4UQaMRblksFgIK23dauwt7OFE4FzZ9L4jdXny+HEZMBblkhaBEFZatcr404OJwFzZ9MeGbQBY98V3HjuHqFkkEQhhpVWroGVLaNPGY4c0Vzbd2TSG/OBQfl2wwmPnEDWLJAIhrLRqFQweDEp57JDmyqYFwaFsb9aWtvullqNwTRKBEFbJzIS9ez0+PuCqEunW5h2IO7Ibioo8ei5RM0giEMIqXhgfgLKVTQF2RHWmwdnTxoBxDTUpKYXYiQto++R8YicuYFKSVF11l8waEsIqq1ZB7doQ79niu+bKpq3Cw7jqjmth/lvGwrKOHT16PqtMSkph5pp0Cl1Miy3Ummmr0wB4KTHO16H5HUkEQlhl1Sro29dY9OVhZWoOFRQYSWfDBhg71uPn87VJSSklN/qKzFyTLonADdI1JIQVzp41bsreXEjmLDTUWL1cQ1YYz1yT7tbrCrUmKVnKcF+M5YlAKRWtlFqmlNqulNqmlHrQ6piE8LoNG4xP6b5KBAB9+hgrjGvAgLGr7qDyTJyTIsngIixPBMB54FGtdVdgAPBnpVQ3i2MSwrscA8UDB/runAkJcOJEjRgwDq7EdNu8gkKmLEr1YjT+z/JEoLXO1FpvLP77SWA7EFXxu4TwcytXGuUemjXz3Tn79DH+rAHdQ+P6R1fq9RmmtRWiNMsTgTOlVFuM/YvXuHhuvFJqvVJqfVZWls9jE8JjtDZaBL7sFgLo1g2Cg41NcPzcS4lx3DEgpqRlEKwUdwyIIcrFGgoH6R4qn21mDSml6gNfAg9prU+Yn9daTwWmAiQkJEgZReG/du+GrCxjRbEvhYZCq1aQ7t5Aq929lBhXZkZQUnIGD83a5PL1k+duKz2TSpSwRYtAKRWKkQSma63nWB2PEF7lpYVkbomJqTGJwJWKbvS5eQU+jMS/WJ4IlFIK+BDYrrX+h9XxCOF1q1ZBw4ZGV42vxcRA2sXn34vAYoeuocHAb4EUpdSm4see0lovsC4k+0pKzmDy3G1lPt0oZXQ9R4WHMWF4Z2kC29mqVcZsoSDvfw5z3pegVXgYH9cKp1N6ujGF1Afn9yTztZT3cx5RN5ScM2U//Qcp4xjyu1GW5YlAa/0T4LnSizVUUnIGz8/b5vIHHIo3n9KajJwzTJxj1FhZvz+b6WvSSjamqhsaxCtjesgvgpVOnDA2o7npJq+fyrEvgaMkdUZuHrMOK545d84Yo2je3OsxeIqra3H8nJt/np8b1Z0JX2ymoLD0UGKRptz3BDrLE4G4OOdfgshT2Yz+ZQXtczKIOHOCxnkniMgz/gzPO8mZ0Dqsje7OxtU92NyqO0HN21MYZBQgO1NQxEOzNvHI7E3c1j9Glt5bYccOI2v36uX1U5n3JQDYV6+J8Ze0NL9KBK6uxbE+wHxTd3z/6OzNZRaelfeeQCeJwA+8NT+FK1JWcEvKEobs3UiwLuJo3UZkhzUip25D9jRuzYawBuSENaTxmeP0T9/K1bvW8gRwslYY61t344d2vZnR61ryQ2pRpGHa6jT2Zp1i+r0+XNAkjEQA0Lmz109l3pcAILNBpPGXtDSjzpGfKG8dQJlrLCiA0FAS46N4uJzZQ67+XQKdJAI727QJPvyQrz74LxFnT5JZvwnv9r+JL+KGsbdxxZ9omp08Rv/0rSVfzy19nzs3zGPysPtYFmvcAFbuzmZSUoq0DHwpNRVCQiA21uunahUeVuYGmtGwOBH40cyhiub/l9p74bPP4K674NVX4cEHXV5/mfcIQBKBrTgGw04eyuKlVf9j9LoFULs2GzoP5H+dr+Sntr0oCgq+6HHCQoM52rAJ87oNYV63IQAM2reJFxa/y8dfPM/iDv144arxpIe3YPrqNBLaNJamsq/s2GEkgdBQr59qwvDOpfrVAc41aMT5sLqE+NHMoclzy18AN2F4ccvqn/+Ehx6C+vXh0UchIcHl9YeFBl94jyjhX9MGarCk5AwmfLGZS9YsZckHf+S69Qv5oP8Y5i/awKlPprGuc78Kk4Cj9EpUeBivjonjtv4xpZ5f1bYX1971Nq9c8QcG7d/Ckg/+yIM/zaBWQT6Pzt4sqy59ZccOn3QLgdFX/uqYOKLCw1AU/2zc1IOQNv41hbSi+f+JvVrBxIlGErjxRti1C9q1g1tvJTEqtOz1j4mTDz0uKF2JKn52kZCQoNfXgHopcGFzjcYnj/HC4ne5ducqtjVrz+PXPsC2Fh2IqBtK8rPXuD11znxs51lDDi1OHOXpZR8yaseP7A9vwb1jJpHeKlZ+SbzA+f8tukEtlj1/PcEPPwSvv25dUMOHQ04OrF1rXQyV0PbJ+S4fDyk8z66jSfDJJ3DfffDvfxslNDZvhgED4NJLYeFCCA6u0u9PTaSU2qC1TjA/Ll1DFpqUlMK0n/dz65bveHrZR9QqLOC1IXfyQd9Ezgcb/zWO6aJlNhpxg/MS/Nvf/5mVu7MBONSwKX+94Qlm9BrBW9+8wYzPnmbc2FeYsqhWQP5yeIt5yqNK20dwwTk2hjWnt5WBxcTAli1WRlAprtYFhJ07y9Rv/ga/roXnn4dnnrnQLO7Z00gKd98NL7xAUuJ4t6eeBirpGrLQlyv38I/5/+D1hW/zS7N2jPjD27w74OaSJOBJ0+8dyB0DYkot2Pi5TU/GjX2FwqBgZnz2NHV3SaleTzJPeYw9dgCA9w57f3ygQjExcOgQ5OdbG4ebnhvVndDgCz+59fPPMGP2JC7dvR7efReeffZCEnC46y6480548UVWvD2t3KmnwiCJwMeSkjMY/Nr3xD84k08+e5ox25Yx5bLfMm7cK+xzMRMoPMxzN42XEuN489ZepWq572nSmnFjX0ErxcyZT3H1Pf9h8Gvfy5iBB5inKbbPNhLB2lpNrQjngujiEs4HDlgbh5sS46OYcnPPkr7+O/avJj5jB2rmTKNLqDz//jdccgnPfPYKLU+UrVgs00gvkETgQ5OSUnh41iZC9+ziy08fo1fmTv46agL/HnQrWpX9rwgNUkwe3d2jMSTGR/HGb3oSFnph4HlPk9aMG/cKRUox47OnqLNrJw/P2sSkpBSPnjvQmKcpxh47wNG6jajb0uKFXDHFEwn8aAppYnwUK58cyt7XruPJyFMQEQG33FLxm+rWhS++oHbRed75+nVCC0t3L8k00gskEfiIY7PthPStfPXpY4SfPcVtY18umd5pFhUexpRbenqlD9M8myRYKXY3iWbcuFcA+OyzibQ/ls701WnSMqiGCcM7l0q47bMz2NektfXTFx2JwI9mDpWyfr2xyY47u5R16sS2F/5Bn4M7+NPPn5c8LNNIS5NE4ANJyRlMX53G6F+WM23WJLLrNuTG3/6dDa1LV590bK6x77XrWPnkUK8OZDl/wioqnla0u0k0Y8e+CsDMmU/RJtuobySqxpxwO+Vk0DTBBrWeWrc2/vTHRJCfDykpxrabbur3+H1kDrmG329eQK3CAplG6oLMGvIyR0vgnrVzmLTsI9ZEX8L4G5/meFiDktdEhYex8smhlsXovAJzd9Noxo19hc+nP8Fb37zBTXdMkYqNVeQ8ZbFzrQIiTucSMbiP1WFBnTrGFpl+0DVknvb5cvRZrigoqFQiAGj5xEMwciQ7+xXALdb9rtmVtAi8yJwEvul8Kb/9zYulkoACy5uoE4Z3LjWbaFfTGJ655o/0ytzJPeu+ktkVVeCYOpqRm4cG6u7dBcDPIRYPFDv4wb4ESckZTPh8c8m/YUZuHt9/WlydvpKJgGuugTZt4L33PB5nTSCJwEscSeDutV+VJIEHR0/gXEjpWUC3D4ix/NN2YnwUtw8ovRL5my6X8W2nQTzy43Tq7NopYwWVVN7U0TfTbfIrFx1t+0Qwee42CopKr4bslvkr2XUbXRjncFdwMNx7Lyxdaqw+FqXY5KeyZrn9/Z+ZtjqNu9Z9zTPLPmR+58E8NOqxknLQYLQE7hhgn1LQLyXGlZ6qqhTPXPNHzoTW4Y0Fb/Lk7I2SDCrBPDUxNvsA+cEhbAxqZFFEJo4WgY0rC7gqLdHj0K+kNI91b6DY7K67jIJ/U6d6ILqaRRKBhyUlZ7BydzZ3rp/Ls9+/z4JOg3hw1IQyi8TevLWXbZKAw+TR3UvNcjlaL4Lnrr6fXpk7+d3qr3j6K5lO6i7z1MT22RnsD29F88b1LYrIJCYGTp2C48etjsRttQvy6ZS1ny0tOlbtAC1bsi1hCMfeeY9OjyURO3GBTJEuZotEoJT6SCl1RCm11epYqsqxUOyhWZv4/YZ5TF46lYWdBvLA6MfLJIE7bNAd5IpjlouzuV0vZ2GngTz64zRaZO6TVoGbruwSWer72GMH2N2kdZnHLeNYVGbj7qGIuqW7Ubsd2UuILmJPmy5VOt6kpBReixlCk7wTDN+5ikKtmbY6TZIBNkkEwCfACKuDqCrngcE7Ns7n+SXvsajjAP5aThKwW0vAWZkEpRTPXP0nzoTWYcqCt3ghyX9q1Fhp2Y4LK1lDCs8Tk5vJ7iatSz3ua44PK+2enM+9y44YD9o4EVzXo2Wp7y85bPTtD//9dVU63sw16fzUthf7w1tw2+aFJY9PW23ffwNfsUUi0Fr/AGRbHUdVPT9vG3kFhdywbRkvLf4Pizv05y83PEFBcOlPNHZPAg7mshZZ9SN47ur76H0wlTE/fiGtAjc4jxHE5B4itKiQPY2jLCtrYJ7FtCmoIQCbVtnz03BScgZfbij9c9Yjcxcnw5swYnglZwwVK9QarYL4rOdwBqal0P7YhRIbgd4qsEUicIdSarxSar1San1WlnWfqswmJaWQc6aAobvW8sb8N1kV08NlEhgc29gvkgDgsqzF3K5D+K7jAB77cRoff7TQxbuEM+cxgtjiGkO7G7e2rKyBeRbT0XrhnAsKIcWmicDVHsVxh34lpXmHqg0UQ0mNrc/jhlEQFMw4p1bBzDX2X1PhTX6TCLTWU7XWCVrrhMhIe/SzOlYM90vfyv99/Rrbmsdy75hJ5IfUKnlNVHgYb93ay6/2Bk6Mj+IO03RSlOLpa/7M2ZBaPDz/P9YE5kecy0s4is0dbN7GsjUj5paIVkEcatCEBlkHLYnnYszxhp07S8dj6axr0r7KxxzX3xgXOVovgkUdB3JzylJqnz8HUGaT+0DjN4nAjqYsSqXb4d188MULHGjYjDtvmczp2nVLnn/r1l5eLxXhLa5aL1n1I5jabwxX7N3Ass++syAq/+FcXiL22AGONmjM07cNsOxnwVVL5GDDSNqesWePbHiZgeI9BOsiMjp0K+cdF+f8Mz2j1wgizp5kROpKAIKq1sioMSQRVEPtPbv47+xnOVGnHr+99UVy6l6YIx4eFuqXCcCZedYGwLT4kZysFcb5V1+zICL/4qjn9Jv6p2nax9oaQ+YCeACHw5vTKT/HoojKl5ScwXHTRjRxh4yB4qG3X1utY4eFGre8n9v0YG9ES27bZHQP1Q4J7FuhLa5eKTUT+BnorJQ6oJS62+qYLio9nRmfPwPAb299icyGF7qrFK772f3Nc6PKXsOJOvWZ3utahqasgD17LIjKz2ht7FPcpWpTHj3F0UJxTu5Z4ZHUOZIJhYUVvNP3Js/dRpHpsbhDv3K4QRNGXFO9Wk1nC4wjaxXEzJ4j6H9gGx2OppU8HqhskQi01uO01i211qFa69Za6w+tjskVx/S73g/MYF+fSwk/d4b7xr3EXqcNZRT2KBvhCYnxUS43xvko4QZjlfTf/25BVH7m6FFjf2AfbVh/Mc43vD11mxBUWMjCxRstjKgs1yuKd7GleWy1j+3cRfblJVdxLiiE2zYtDPi9CWyRCPyBY1OZ44eP8cnnz9Ei5xD33vQsXa8fUlJmOCo8zJYrhqvDvNoYIKtBE77sfiX573/IgqWbLYrMvpzn6/9p0nTjQYtbBFB2Js7BBkYrds7Xq60KqQxXU5Pr5Z8h9tgBUqq6otiJcxfZsXrhLOo0kDHbvueJK9pU+9j+TMpQu8FRQK52QT4ffPkCXY/sZfyYSfzYqht7dmRZWkLa2xwtmymLUsnIzUMBGpja/yZu3bKYtOdeI6nx32tEC8gTzBvWN0gzus++KwznGisDo+xMnIMNjUqotQ7aZ8tKV5Vuux/ZQxCavVVcUezM+ef5YG4eSwaPYtSOHxm9bx0MqH6Lw19Ji+AiHAXkQgrP887c1+mXvo1Hr3uYZbF9gcDY99Qx6BkVHoZjkt3exlEs7DSQcevn8e+5yZbGZyeuqo6eDanFC5tPWRiVwdz9cbBhMwC6FuRaEI1rrn6f4jJ/BWDkndd75ByOn+c3b+3FhvbxpDdqzrpJUwJ6oaQkggpMSkph5e5slC7i9W//ydW71vLc1fcxt9sVJa8JpL5F8y/puwNuplH+aa5Y8ZVFEdmPq6qjeyNakXEi36KILjDPHDpVuy4natdjRKOyffJWcfX7FHd4F5mNIrl2WC+PncfRcjtwIp/ZccPou3sj73y4OGCTgSSCciQlZxg1SLTm2aXvc9O2Zfz9sjv4tPeFTyV22FTGl8y/pFtadmJlmx6M3/C1sYWgcFF19AC7m0Tb4gODeevMqPAwdHRrYvPss5bAVVG+uEO7yOni2XE355bbF3HDKFRBjN64MGA3YZJE4IJjYBjgwZUz+cOGebzfN5F3Bt5a6nU1ZXaQu1zNRX+3/81EnjjGnEdftygqe3H+N6p1voCY3MOkNbXBhvXFnPeqXvnkUBp1irVV4TlzUb4G+aeJzc7gx0ZtPXoe55ZbZsNIfmgXzy0pSziUbX0XnhUkEZg4BoY1MH7Nlzy8cgaz44bx8pV3l6px4i8F5DzJ8YmybuiFH5sf28aztXksPWe+zzNzZAaR86fuNrmZBOsi+l07yL4fGGJibLV3sblr7ZJDuwFY1cizs3rMLbRZPa6h5alj3Hhkm0fP4y8kEThxJAGAu9d+xVPLP2Zu18uZOOKvpZKAPxWQ87TE+CjyzzvVZVGKd/vfRGz2AY5O+9y6wGzE8al78cjmACSMGGRxRBWIjjbWOpw5Y3UkQNkbdNwhY6A4q7Nnf9/MrdulHfpxrG4jHkn/0aPn8ReSCIo5J4E/rDe2mPym86U8fP2jpbaYvGNAjF8VkPMGc4GubzsPZn94C+5Z86VFEdmH8xqC996bbzzYqZO1QVXEsfevTVoF5ht0j0O7ONCoOePH9PPoeczjJc2aNCT7prG0+mExHD7s0XP5A1lHwIUqogC/3fgNzy01tph03mdYYWwvadsmvoUKg4KZ1mskTy//yNgYvEMHq0OyhHkNQdOD+8hs0JQ1vx4nMd4mW1SaORJBWpotVj+b5/n3OrIbldDHK793ifFRpY+7vSV8+h58+ik89pjHz2dn0iLA+KHTwO3JC3hx8bss6jigzBaTgTYwXFnfdL0MgF/efN/iSKzj2KDIIfbYAXY1bm3vmSiOLStt0iIApwHtJwbROvsgUcMu882Ju3aFQYPggw+MGlEBRBIBxgDV2E0Lefm7/2Nxh3785YYnSiWBQBwYrkiUi6mQmQ0jWdu6GyGfzw7IudhJyRnkOFfM1Jr2xw6wp4l1u5K5JSrKGP+y0cyhEhs2GH8mVG1Hsiq55x5ITYVVq3x3ThuQRAD8acdiXlv0Dktj+/LnGyaW2l1MkkBZrqaRAszrejmdsvbxxX8DbwezyXNLzzaJPJ1Lw3NnLN2VrCIlYxnPLiarQWP2J++wOqSy1q83/uxTvYqjlXLLLVC/vtEq8DHn8aXBr33v0w9UAZkISv7Bn/iGD4bdyYSv/8n3Hfvzp8SJnAsxkoBCkkB5HANtZt92HkyhCmLA2sUWRGUtc8XM2Gyjq2VPY/usIXAw71+cXr8pGSk77deS27EDWrWCiAjfnbN+fRg3DmbPhhMnfHZa8/9JRm4eEz7fTPwL3/kkMQRcInD8gx/MOc3kJe9yz9L/MqfH1Sx/9T80bdqoxlYR9bTE+KgyXURH60WwKqYHo1N/CLg+VrP22cYv7e4mrW03tuSqCmnz40fsN5axZw/EWlAI7u67jem0n33m1dM4twAenb25zB7NBUWanDMFJYlh4pwUryWDgEsEz8/bxvmzZ/nX3Cn8fuN83u03hkdGPMDSX3NKrbi02y+vHU0Y3pnQ4NJ7/M3tdjkx2Zksn/GtRVFZw7zVYftjB8gLqc2Rhk2sCagCZauQRhJ1IouDOdauJTB3jeSl/grtq75HcZX16weXXOLV7qFJSSk8NGtTSQvAnT2T8woKvZasAyoRJCVnkJ97gg+/eIFRO37klSv+wGtX3gVK2XtAz6YS46OoV6v0DORFnQZxLiiEg+99Yk1QFiky/R63z85gb+NWFNrwV6xsFdJI6pw/R9da1hWfS0rOYMIXm0tujEePHifsyCG2hzX1fTBKwf33w7p1sNDz410ldcyqwFv3KVv8lCqlRiilUpVSu5RST3rrPFPnrGXGZ08zaP9mHhv5EFP731TynB0H9PzBcVPf+Ik69VnRvjdXbFoGRYGz/Z+5m6x9dgZ7Grd2OcPKaubB/szifQkmdLUu1ufnbaOg8EI2bX3cWNQ17UjZSQneYG6NfN13pNEt9dhjcP68R89VnU/13rpPXTQRKKWWKKV6euXsxvGDgX8D1wLdgHFKqW7eONf4Of+kS9Y+7r/xab6IG1bqObsN6PkLVz+Y87peTquTRwNqCp652Fz0cXsVm3NmXlVbEGWsJbiy7lnLYsoxbVYfnXsIgO1hZauRepqrgdon56Wy5v4nYNs2+Ogjj54v4yKf6hUQHhZapts1LDTYaz9P7rQIHgfeVEp9rJRq6YUY+gG7tNZ7tNbngM+AG7xwHt6/8QFuu/VllnTsX+rx8LBQGROoIldTSVd2HcT5OnW8PthmJ84315jcQ0axuREDbftz5VyF9KNni1vGFi0qczUAGl3cIkgPb+H185sHz8Hoj3+koD1cdhk884zHZhBNSkqp8PmIuqHsfe06Nj13DVNu7lmqZPirY+K89vN00RITWuuNwFCl1E3AQqXUHOBvWmtPdVZFAc4/gQeA/uYXKaXGA+MBYhzL4ivp3pv6M3FOXXD6Tw8LDWby6O5VOp4oWxKgVXgYE4b3ImTvKPj8c3jrLQgJjEomJSULkk7Ah5BwzQCrQ3JP06ZQp45li8pcdZXE5B4iL6Q2BU293yIor9894/hZeOMNY/D49dfh5ZerdR7nUjauBAcpnht14V5UpgSGF7n1G6qUUkAq8B/gJeBepdRErfWnHohBuXiszBC61noqMBUgISGhSnMTXd+0Otv2U5u/cPkDO3askQiWL4dhw1y+r8ZKLb6x2aB2T0WSkjNKfhdW1GtC8OYdWPGb4OpGHJN7iLTw5ky+4RKvn79VeJjL7hoFJIW0IvH22+Ef/4D77rtQm6kKHKVsyvPGLT0tuxe5M0bwE5ABvInx6f1O4Aqgn1JqqgdiOABEO33fGjjogeO6ZN6YQ5KAl1x7LTRoEFDdQyVSU6FFC2jY0OpIymXuF9/XoBnZW1MtWVTmapwpJvcQmY1b+eT3c8LwzuV+Gp2yKBVeecV44KmnqnWeisYGosLDLL0XuTNGcD8QpbW+Wmv9jNb6G631Lq31XwFPVINaB3RUSrVTStUCxgJzPXBcYaWwMEhMhC+/hHPnrI7Gt1JTbd8aMPeL749oSXROpiWLyiYM70yo80IMrYk+fpjoPl6ZM1JGYnxUuZ/UD+bmGa2ARx6B6dNh7doqnSMpOcNlsgF7bHl70USgtd6qdbmrHa6rbgBa6/PAX4BFwHZgttY6MLcJqmnGjoXcXPjuO6sj8a2dO+29BwFlu2P2h7cg/OwpTh3KKucdXuZ0l2ycd4L65/I4HVX1bpjKKm+ab0lr5cknoVkzePTRKq2ar6hbyA6Vjau1jkBrvccTQWitF2itO2mtY7XW1RuREZZzzMnutOwcx8MakP5/H1odku9kZxs7ftm8RWDujkkLNyYEdj/j+0QwZVFqqTUEMcVTR321hgBcz35TwJVdigerGzSAF1+En36COXMqffyKFoLZoZSNLRaUiZrDue/5XHAo8zsNovGShcz9eZfVoXmN82Kk8U9PNx60eSIwd8ekFU/TjMw64PNxAvNN0pEINoU09lkMifFR3NQnqlT3jQa+3JBx4d/jrruM0hOPPw5nK7fmoryFYHZZcCiJQHiUue95UadB1Cs4y/KpX1gYlfeYB10bphmN5MWFjawN7CIS46OoX+fCpEFHIojK9v04gfkm6VhMdr5NW5/GsWxHVpnum1L1fUJCjNlDe/bAqFFw+rTbx3bV4vDmArHKkkQgPMr86W5NdHfyg0PpvnW1RRF5lznxtc8+QEFQMC9utW6VrrtynVbznqkVRla9cNrkZPq87pb5JhmTe4is+hE8MMprBQ1cKu+6Sz1+9dXw8cfw/fcwfDgcP+7Wsc2rub29QKyyAmOlj/AZ85zss6F1WBN9CUPTki2MynvMN4/22Rmkhbck/aT9Z0o1CgsttY/C/vCWtMnNpFFYaAXv8jzz+p6Op48QFBvr85tkeesJynTr3Hkn1KsHt90GV11lFKZr6ro4nvNaDTuvW5IWgfAoV03gFe160+5IGosWVG3qnZ2ZbxLtsjPY0zjKL4oYKtN8xv3hLYjJPVTmcV9wXt/T+3wOTeK6+DyGSnXf3HILJCXB1q1wxRWQmVnmJa5qGHlzT4HqkEQgPMrRBA53+lT5Q7veAKz893Rb/hJUh/PNI6iokLY5maRFRtum77ciuaZCb2nhLWlx8hhnTrjf911dZap+rt0LBw5Ysg9BpbtvrrsOvv0W9u2Dyy+H/ftLPV1eDSPbbQCEdA0JL0iMj2LKotSSbodfm8ZwsEFTBuxaz8uLUm3ZNK4q526N4H17qV1YQO+r+xPvB9do7grZH9GSIDS9i9zr964uxx4EjqmjGbl5/OvjpdxQVGTNhjRUob7PlVfC4sXGSvrLLoP//Q+GDCFp08FyVxLbce8TaREIryj1S6AUP7TrzaX7N3Mo+5R1QXmBcx9w3/wjAMRf7R/F5sxdIY6ZQw+09c1twbwHAUCr7OIuFosSgcOkpBRiJy6g7ZPziZ24oOKqoQMHwrJlUFAAV15JbreerHz+n4QUut7HwI7dhpIIhFeY+5l/aNebhvmnic+0X7O4qsx9wI3S9gKwIL+BtYG5ydwVkt+mHQADyfXJ+c17EMCFNQRWJoJJSSlMW51Wsn1kodZMW51WcTKIj4fdu+G99zh+LJcpX0/hx3fv5r41X9Dw7IUPP3aaMupMuoaEV5hX4f/UtheFKojL9my0JiAvcDV19Hjtery89igjr7IwsEoo1RWiNfyzvnFDs0h07iHyg0Op3dIbW5+4Z+Ya1/syzFyTXvEq4Lp1Yfx4rtjdkiF7NnDPuiQmLv+EB1Z+xvL2fdjaogOX/uYaBkfX9lLkVSeJQPjEiTr12dSyE0P21pxEYO7rbZdjbE958Lj91xC4pJSxPaOPEkG4afoqGC2Cg+HNaRdkXWdFeRvJu7PBPEBQUDDLY/uyPLYvXY/s4c718xiQnsJ1qSthxX/hz0B0NPTpA127GjWMIiMvfDVrBk2aQO3aZZvWXiKJQHhFRN3QMk3/H9r15sFVM+HYMeMH3c+ZB1vbH8tgVduetuwDdltsLPzyi09ONXl0dx6ZtQnnna1jjh8mrEtHn5y/PMFKubzpB7t5U3Z+7/Zm7Xli5IMANDx7ii2jI2HjRuNrwwaYNw8KC8s7lJEM6tQp/TV1qjFLyYMkEQiveG5U91IzQgBWdejDwytnGLMsxo61MDrPmDC8MxPnpJBXUEjdc3m0PHWMtKb+MXW0XLGxMH8+FBWBDz6VBwcrihw/I1rTJjeTrKhLvX7eiozrH800FzuJjesf7eLVpSUlZ5SbSBq0iIShQ40vh6Iio0JvVlbpr+xso56Rq69Gni9fIolAeIWr3eBuv/lmmPsSLFpUIxKB8zVG7DCK6g0eOZB+fjB1tFzt20N+PmRkGN0XXmSuOtro7Cka5J/h41NhPODVM1fMMQ4wc006hVoTrBTj+kdftEqoY/KAqyRQ7iBxUBA0bmx8WVioUBKB8BqXc7KvvtpIBFr7rP/Tm0quceZh+C/0GzHI6pCqxDENtn3yUT4Fflq4mkvv9W4iKK/q6Nba1ncbvpQYV+ny0BPnbCGvoKjM48FK2aqukCsyfVT4hGMF6eMnW0BmJt/PXmJ1SJ6Vmmoktg4drI6k0pynwe6LMGbrLPx6lddXgZvHUhyJ4Gx0G6+e1xsmJaW4TAIARVrbOgmAxYlAKXWLUmqbUqpIKZVgZSzCe5xvND+0NcpNbHh/Vs0qN7FzJ7RpY2zR6Wecp8EebBhJQVAwLbIPer0UQpmqo8eNRPCbWzw7EFpd5jIYrn5uZ6wpO6bg4A+TB6xuEWwFxgA/WByH8CLnG82hhk3Z0bQNA3ett2XNlSrzg32Ky+PcRVMYFExGw2Y+KUdtXtDW5cxR8iOacP2l9vl3dKdw3KSkFIoqmFnqD5MHLE0EWuvtWusadDcQrphrrvzQrjd9D2wj+0iORRF5mNZ+sU9xecpuW2lUIfXFJ1nnqqM3NMqndid7da1drHBcUnIG013MMHIIUti+WwisbxG4TSk1Xim1Xim1PivLog22RZWY51//0K43tQvPMyi9giX7/uTgQTh1ym9bBOYumv0Rxr4EPv8ku2eP5TWGzCrarCYpOYNHZ28ud1N6gNv6x3gnMA/zeiJQSi1RSm118XVDZY6jtZ6qtU7QWidERkZ6K1zhBebpdOuiu5MXUptLa8oq49TiRq2fJgJzF01uy2jCz54isW1d3wVx/rxRxrldO9+d0w3ltYo08NCsTRWuNg4LDbLFxvTu8Pr0Ua31MG+fQ9hblGkFbn5ILVbHXMJV+2vIrmV+ngjANNU3qQAWvGeUmkjw0RyO9HRjha3NWgTOiwYrQwGvjunhnaC8wG+6hoT/crXz088dEog5egD27rUoKg/audOYLRRl/75gt8TGGn96seaQeSbOT4uKd6+zWSJwbi25SwG3D4jxi7EBB6unj96olDoADATmK6UWWRmP8A5XOz/1u2+c8eTixZbG5hGpqcZAsYWF0jzKcTP2UiJwNRPnu3mrSp/bRhwD2u4sfwxWijdv7eU3XUIOlq4s1lp/BXxlZQzCN8qsMtYaWrWCpUth/HjrAvOE1FTfdaH4Qr160KKF1xKBq5k4LbIPcj4omJDWrb1yTk8ob3N7Z2/8pqdftQQcashHGOFPkpIzGPz6MuZEdCZn/iKSNriu/+4X8vONPWv9eHzApfbtvZYIXM3Eick9THqjZhAc7OId9uCqi9PZ4NjGfpkEQBKB8DHnboGf2vYi4vRxPnlvnv+uMt61y6ggWdMSgRf3JXA1Eyc69xBHmtr7JlreeEGwUtwxIIbp9w60KLLqk6JzwqecuwVWtukJQMKuDUxZ1MU/P005Zgz56WKycsXGwrRpRtnjOnU8emhXM3HaHD9EzqX23+u50pvb+wlpEQifcu4WONygKbsbt2bw/s1eL2fgNTU5EWhtdHt5mHnyQOfahYTnnaRdP/8aYK1JpEUgfMo84LayTU9u2rqUmPp++qO4cSO0beuVzUIs5TyFtEsXjx++1Cfr5GR4AVvOGAoU0iIQPmUecFvZpif1Cs7yYqszFkZVDevWQd++VkfhMY75/Qkz9gCwZcUG75/UsZZEEoFlJBEInzIPuK1t24MiFDtnfO1/A8ZZWUZZhBqSCJwH8o/WbcSpWmFsWr7R+/8ve4ykI4nAOn7aHhf+zNElMOHzzeTUrk9Kiw70SF3H7Z9vLvW83Th28XJsvfn3egcYCNCvn9WheUSp+f1KkRbegqjsgzy7KNW7/yd79kBERM3rXvMj0iIQlpg8dxsFxUXcV7XpSfzBVELPnmHy3G0WR+aaq9WwGz7/Dq0U9O5tdXgeYR6w3x/e0if7ErB7t+2KzQUaSQTCErl5BSV/X9mmJ6FFhfRL31bqcTtxtRq2W0Yq+yJjoEEDi6LyLPP8/v3hLYg+fpiohrW9d9KiIli/Hnr18t45xEVJIhCWW9e6G/nBoQzav9nqUMpV5lOx1vQ49Cvrm3W0JiAvMA/kp0W0pHZhAc/Ee67LxlxsbslXP0B2Ngwe7LFziMqTRCAs4bxXTX5obTZEdWXw/s0odyp7WcD8abn1iSM0PXOc/e27WRSR55nn959ubWwiPzzstEeO76p7bcXHScaTgwZ55ByiaiQRCEuY9/NY2aYn3Y/sIeL0cWsCuogru5TeDKlH5q8A1POD1bCV4bx15D+fvtl40EOlJibP3Vame63H/m3k1m1Y80p0+BlJBMIS5notq4rLTVx3dIcV4VzU/C2Zpb7vkbmT/OAQPj5ZM8YHXIqOhpCQC9M7qyEpOcPl+E+fjF9Y36oLtm0KBghJBMIS5v7oLS07crJ2Xe7Kr/5NxxtyzpS+ifU89Cvbm7XjyLmaeQNLSs5g8N9/YF/9SJZ8s6raawkcm707a3I6l/Y5B9nZwX928qqpJBEIS5j7o1s0rs+pAZfSbvNqq0O7qKCiQuIO7WJLixpWX6iYc19+WngLIo8Y31cnGbiagtonYzsAl9wyssrHFZ4hC8qEZcpUcqy7FR580Ch01ratVWG5FB4WWtK10T47g/rn8tjcshPhYaEWR+Z5zlNldzWJZtzmRRTl5TGlGgvLXG3q0jtjO+eCQ7n8tmurHbOoHqu3qpyilNqhlNqilPpKKRVuZTzCYlddZfy5dKm1cbhwfc+WJX/vWTxQ/EtUJyaP7m5VSF7j/Ol9Rfs+hJ3PZ+D+LdVaWOZqU5d+B3dwsnsPj5e5FpVnddfQYuASrXUPYCcw0eJ4hJW6dTO2SLRZIkhKzuDLDRe6RXoc2smpWmEkDB9g23IY1eE8VXZ1TBynQ+swdPc6lxvKuMvcFdiuXjA9D++iyTVXeiBiUV2WJgKt9Xda6/PF364G7LthqfAqx/aVSY27cGzeQpI2HrA6pBLmVcU9M3eytXks3/+abWFU3uP86T0/pBY/te3FsN3rmHBN9cZEnKemLhvagOCCc7KQzCasbhE4uwv4trwnlVLjlVLrlVLrs7KyfBiW8DbnwcmVbXrS5FQOH733jW2qkTp3iYQWFtD1yF42tezkv5vpXIT50/vGuMG0OnGExNAcz51k5UrjT1lIZgteHyxWSi0BWrh46mmt9dfFr3kaOA9ML+84WuupwFSAhIQEXd7rhP8ptX1lW8f2lRt5fl57W3S9OA90djmyj9qF59nSslO1ukrsrtRAfmZvmPMGfPMNxHloF7GVK6FjR2jWzDPHE9Xi9RaB1nqY1voSF1+OJPB74Hrgdq3N601FIHD+ZH2wYTN+bRLNlbvXkXOmwBatAueukp6HjIHi1OguTBgeIKthW7aEhASYN88zx9MaVq2SbiEbsXrW0AjgCWC01tpPt6gS1WX+ZL2kQ38GpKfQIP80z8+zviy1c1dJz4M7yakXzl//cJUtWis+c/31sHq1sRlPde3cCUePSiKwEavHCN4BGgCLlVKblFLvWhyPsID5k/XiDv0JLSpkyJ4NZVb0WsUx0HmLziRiyCASewfYvIZRo4xP8t+WO4xXLnPF0Y0zvjGekPEB27B61lAHrXW01rpX8df9VsYjrGH+ZL2pVSeO1m3EsF1rLIqoHKdOwfbtNWZrykqJj4dWrYxxgkpwVXF0z9zFnGvYCLp08U6sotKsbhEIAVBqhW5RUDDfx/blyt3raVrLRrV8Nm40NlKpIVtTVopScN11sGgRnDvn9ttcbejTK20b61p2gSC5/diF/E8IW5g8ujuhQRdu+ks69KdR/mnean3KwqhM1q41/gzEFgEY4wQnTsCPP7r9FvMU2/C8E3TIPsDK5gEy0O4nJBEIW0iMj2LKLT1LylOvateb/OBQMv432xYzhwBYtw7atIHIyIu/tia66iqoXbtS3UPmiQCOQnP7usR7NDRRPZIIhG0kxkeVTNU8FVqbn9r2YuDWn5j45RZ7JIN16wK3NQBQr56RDObNK7uzUDnMNYYSDmznXFAII+683ltRiiqQRCBsxblPeUmH/sQcP0zrzL0u69l7m/Nsl5HPfAV79wbm+ICz6683dizbudOtl5tXKQ86vINT3eIYPbCDd+MUlSKJQNiKc5/y0ljj0/fVu9b4vJyDebZLsx1bAPgpvK1P47Cd664z/qxE91BJjaHnh9Ez81caS6E525FEIGzFuU/5SIMmbG7RkWG71vi8nIN5tkuPQ79ShOL5gzW3rIRbYmKgRw+3Vhmb1w+smLkI8vNlIZkNSSIQtmLuU17SoR+9Du5kUp8In8ZhboH0ydjO7iat2ZVno+msVrn+evjpJ8gpvwidq/UDa6bNNZ6URGA7kgiErZj7lLf0HkIQmmvTNvo0DucWSHjeCQbt38zS2L41utBcRZw/3Y/PaQmFhcaagnK4Wj/QI/0XDjRuBc2beztcUUmSCITtONet/+9b9xjdEXPn+jQG55bJyNSVhBYVsqjH0MApNOfE/Ol+Sf0YjtVtRPr/Zpf7HnOLqunpHPqnbWVNK1lNbEeSCIS9KQWjR8PixXDGd3UJnVsmN/yygn2RMfz+jzcEVqG5YuZP90VBwSxrn0Cj5UuMshsumFtUn856hjrnz/Ht5Td5PV5ReZIIhP2NHg15eT7fwjIxPoqVt3ekf/pW2v71nsArNFfM1YytLy+5inpnTxv9/fv3l3ne0aJqkH+a/81+lvbZGfz5N89x/T03+CJkUUmSCIStJSVnMGRNESdrhTH3xXd9srDMuT/8nfteNh4cN87r57UrV+MiP7fpwfixLxhJoG9fY/DYSWJ8FH8b0Z7pX71AlyP7ePqOyYx+7HcB2aLyB5IIhG05+qb3nzrPivYJDPxlFU99udmrycDcH37FxiVsbtWZpJOBOUgMxqd7VzeKpTG9eOvl/0FEBAwdCh99dOHJvDxGTbqPHgd2UGv2Z/z9k6ckCdiYJAJhW85904s79CPydC6d0nZ4dZWx8zljj6VzyeHdJHW93JKVzXaRGB8F5cyafftAiLFhzRVXwN13wyOPGGM5Y8bAihXw3//CTTIuYHeSCIRtOfdNL2+fwHkVxDAvrzJ2PvboX36gUAXxTZfLauxG9e4qKqe0UKHWRotgwQJ44AF4801o2xYWLoT334fbb/dpnKJqrN6q8kWl1Jbi3cm+U0q1sjIeYS/OfdPHwxrwc0wPbty6jFpFhV7rHio5p9aM3r6cn2PiyKrfOGDXDzgEK9dNgpLHQ0Lgn/+EqVONgf233zZaCMIvWN0imKK17qG17gV8AzxrcTzCRsyrjD9JGEXUySxGbP+BiXNSvJIMHOfscehX2uVk8nW3IYSFBgfk+gFn4/pHu3y8Vogq/f9w771w/Dj85S8+ikx4QoiVJ9dan3D6th7gXm1bERAcg4uPzt5ModZ8H9uXXY1bM37tV3zd7QqmLEr16ABkUnJGyRhB4i8ryA8OYUvfq3g1MS7gBzpfSowDYMaatFLdRHkFRUyckwIY/1+TklKYuSadQq0JVopx/aNL3ivsy+oWAUqpl5VS6cDtVNAiUEqNV0qtV0qtz8rK8l2AwlKJ8VFGPzSgVRBT+42h+5E9DN6/mQwP9tsnJWcw4QvjmEFFhVy340dWxPblj4l9Aj4JOLyUGEfLRmW7yPIKCpmyKJVJSSlMW51W8v9VqDXTVqcxKSnF16GKSvJ6IlBKLVFKbXXxdQOA1vpprXU0MB0otz2ptZ6qtU7QWidEBuoOUQHKuX86qfuVHKkXwX1rvjS+91D30PPztlFQaNzA+qdvpfmpbJK6DuH5eds8cvyaorxB84zcPKatTnP53Mw16d4MSXiA1xOB1nqY1voSF19fm146A5B5ZqKMQqfdsM6FhPJJn1Fcvi+Zrkf2eGxaZ86ZgpK/3/DLCk7VCmNpbN9SjwvXi8suptDN3cyEdayeNdTR6dvRwA6rYhH2FWW6+UyLH8np0Drcu/Yrj0/rrHW+gGtTV7Ko00DyQ2t79Ng1gXkA3x3lzTgS9mH1GMFrxd1EW4BrgActjkfY0IThnUutZzpRpz6zelzDqO0/0IuT1T6+c/fSkL0baJR/mq+7DgEgPCy02sevSRzF+Cpzcy9vxpGwD0sTgdb6puJuoh5a61FaaxvsUC7sJjE+itsHxJRKBh/2TURpzZSMZdU+vnP30g2/rOBo3UasbNsLgMmju1f7+DVNYnwURW5299QNDZJZQ37A6haBEG55KTGON2/tVbJhDW3akDl8FB2SZkBubrWO7ehe6n5oF9fs/Jl5XS+nMMjo/pAZQ665M1YQGqR4ZUwPH0QjqsvSdQRCVEZifFTpG/PwCOj9Nbz3HjzxRJWP2yo8jGNZOfxz3t85VrcRbw2+DSg7NiEumDC8MxM+30xBObUnwsNCmTy6uyRSPyGJQPiv+Hi46iqjtMFDD0Htqg3uThjemTP33k+H7APcdutLHA9rIKuJL8Jxg588dxu5ecbMqoi6oTw3Sm7+/kgSgfBvEybAiBEwcybceWeVDpF4cBNs+IYZl93Cz22N7qcJwzvLDe0iyrTQhN9S2g/n+CYkJOj169dbHYawA62hVy+j0NmqVdC0aeXef+QIxMVBy5awZk2VWxVC+AOl1AatdYL5cRksFv5NKaP0cVoaDBkCGZWYeKY13HWXUSRt+nRJAiJgSSIQfm1SUgqxi88ydsxznNq1l+w+/WH3bvfe/O67MH8+/O1v0F2miYrAJYlA+C3nImerY3pw29iXUcdPcLLvANi6teI379gBjz4Kw4fDX//qm4CFsClJBMJvmYuZbWnZid/c9hpnCorg8sth7dqybzp6FJKS4NZboV49+Phjo3tJiAAms4aE33JVzOzXyDbcfPvf+HHxK8bU0k8/hXPnjP1zV6yAbcXVROvVg1mzjEFiIQKcJALht4KVcpkM0sNb8O1/PufaR34HN95oPFi/PgwezC9XXM+/zrfi+wYxRKaEMaFVhkyBFAFPEoHwW+P6R5dbA/+RH49Q9M5nXLdvPXTrBr17k5RymIlzUsgrKASMGvrOu2sJEahkjED4rZcS47hjQIzL5/IKCvnzwv0MPhRDUmgUhISUbENpfp2n9jQQwl9JIhB+7aXEOCoa6s3IzeOhWZvo9PSCcre29PSeBkL4G0kEwu+5UwnzXGH5K+irsuuWEDWJJALh9yqza5a59SDF5YSQRCBqAMeuWe6UjdZQsqdBVHgYr46Jk4FiEfBsMWtIKfUYMAWI1FoftToe4X8clTCTkjN4aNamcl8XFR7GyieH+i4wIfyA5S0CpVQ0cDXgeh6gEJWQGB/F4NjGLp8LUkg3kBAuWJ4IgDeBxzFa7UJU2/R7B3LHgJhSlSPqhgbxj9/0km4gIVywdD8CpdRo4Cqt9YNKqX1AQnldQ0qp8cB4gJiYmD779+/3XaBCCFEDlLcfgdfHCJRSS4AWLp56GngKuMad42itpwJTwdiYxmMBCiFEgPN6ItBaD3P1uFIqDmgHbFZGG741sFEp1U9rfcjbcQkhhDBYNmtIa50CNHN8f7GuISGEEN5hh8FiIYQQFrLFOgIArXVbq2MQQohAZOmsoapSSmUBVZ021BQItO4nuebAINccGKpzzW201pHmB/0yEVSHUmq9q+lTNZlcc2CQaw4M3rhmGSMQQogAJ4lACCECXCAmgqlWB2ABuebAINccGDx+zQE3RiCEEKK0QGwRCCGEcCKJQAghAlyNTQRKqRFKqVSl1C6l1JMunldKqX8VP79FKdXbijg9yY1rvr34WrcopVYppXpaEacnXeyanV7XVylVqJS62ZfxeZo716uUukIptUkptU0ptcLXMXqaGz/XjZRS85RSm4uv+Q9WxOlJSqmPlFJHlFJby3nes/cvrXWN+wKCgd1Ae6AWsBnoZnrNSOBbjG1sBwBrrI7bB9c8CIgo/vu1gXDNTq/7HlgA3Gx13F7+Pw4HfgFiir9vZnXcPrjmp4DXi/8eCWQDtayOvZrXfTnQG9hazvMevX/V1BZBP2CX1nqP1voc8Blwg+k1NwD/04bVQLhSqqWvA/Wgi16z1nqV1jqn+NvVGBVf/Zk7/88AfwW+BI74MjgvcOd6bwPmaK3TALTWgXDNGmigjDLG9TESwXnfhulZWusfMK6jPB69f9XURBAFpDt9f6D4scq+xp9U9nruxvhE4c8ues1KqSjgRuBdH8blLe78H3cCIpRSy5VSG5RSv/NZdN7hzjW/A3QFDgIpwINa6yLfhGcZj96/bFN0zsOUi8fM82TdeY0/cft6lFJXYiSCS70akfe5c81vAU9orQuVcvVyv+LO9YYAfYCrgDDgZ6XUaq31Tm8H5yXuXPNwYBMwFIgFFiulftRan/BybFby6P2rpiaCA0C00/etMT4tVPY1/sSt61FK9QA+AK7VWh/zUWze4s41JwCfFSeBpsBIpdR5rXWSTyL0LHd/ro9qrU8Dp5VSPwA9AX9NBO5c8x+A17TReb5LKbUX6AKs9U2IlvDo/aumdg2tAzoqpdoppWoBY4G5ptfMBX5XPPo+ADiutc70daAedNFrVkrFAHOA3/rxJ0RnF71mrXU7rXVbbZQ5/wL4k58mAXDv5/pr4DKlVIhSqi7QH9ju4zg9yZ1rTsNoAaGUag50Bvb4NErf8+j9q0a2CLTW55VSfwEWYcw6+EhrvU0pdX/x8+9izCAZCewCzmB8qvBbbl7zs0AT4P+KPyGf135cudHNa64x3LlerfV2pdRCYAtQBHygtXY5BdEfuPl//CLwiVIqBaPL5Ant5zsdKqVmAlcATZVSB4DngFDwzv1LSkwIIUSAq6ldQ0IIIdwkiUAIIQKcJAIhhAhwkgiEECLASSIQQogAJ4lACCECnCQCIYQIcJIIhPAApdQypdTVxX9/SSn1L6tjEsJdNXJlsRAWeA54QSnVDIgHRlscjxBuk5XFQnhI8W5g9YErtNYnrY5HCHdJ15AQHqCUigNaAvmSBIS/kUQgRDUV7ww1HWPXqNNKqeEWhyREpUgiEKIaiks9zwEe1Vpvx6iEOdnSoISoJBkjEEKIACctAiGECHCSCIQQIsBJIhBCiAAniUAIIQKcJAIhhAhwkgiEECLASSIQQogA9/9gBtSqvaYEdwAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "X_grid = torch.from_numpy(np.linspace(0,1,50)).float().view(-1, d)\n", "y_hat = neural_network(X_grid)\n", "plt.scatter(X.numpy(), y.numpy())\n", "plt.plot(X_grid.detach().numpy(), y_hat.detach().numpy(), 'r')\n", "plt.title('plot of $f(x)$ and $\\hat{f}(x)$')\n", "plt.xlabel('$x$')\n", "plt.ylabel('$y$')\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "cleared-terry", "metadata": { "id": "48dcIO6l4sqc" }, "source": [ "# Things that might help on the homework\n", "\n", "## Brief Sidenote: Momentum\n", "\n", "There are other optimization algorithms besides stochastic gradient descent. One is a modification of SGD called momentum. We won't get into it here, but if you would like to read more [here](https://distill.pub/2017/momentum/) is a good place to start.\n", "\n", "We only change the step size and add the momentum keyword argument to the optimizer. Notice how it reduces the training loss in fewer iterations." ] }, { "cell_type": "code", "execution_count": 17, "id": "ordinary-print", "metadata": { "id": "mrCJavAz4sqc" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "iter,\tloss\n", "0,\t3.47\n", "150,\t2.94\n", "300,\t0.83\n", "450,\t0.55\n", "600,\t0.12\n", "750,\t0.10\n", "900,\t0.06\n", "1050,\t0.03\n", "1200,\t0.00\n", "1350,\t0.00\n" ] } ], "source": [ "# feel free to play with these parameters\n", "\n", "step_size = 0.05\n", "momentum = 0.9\n", "n_epochs = 1500\n", "n_hidden_1 = 32\n", "n_hidden_2 = 32\n", "d_out = 1\n", "\n", "neural_network = nn.Sequential(\n", " nn.Linear(d, n_hidden_1), \n", " nn.Tanh(),\n", " nn.Linear(n_hidden_1, n_hidden_2),\n", " nn.Tanh(),\n", " nn.Linear(n_hidden_2, d_out)\n", " )\n", "\n", "loss_func = nn.MSELoss()\n", "\n", "optim = torch.optim.SGD(neural_network.parameters(), lr=step_size, momentum=momentum)\n", "print('iter,\\tloss')\n", "for i in range(n_epochs):\n", " y_hat = neural_network(X)\n", " loss = loss_func(y_hat, y)\n", " optim.zero_grad()\n", " loss.backward()\n", " optim.step()\n", " \n", " if i % (n_epochs // 10) == 0:\n", " print('{},\\t{:.2f}'.format(i, loss.item()))\n", "\n" ] }, { "cell_type": "code", "execution_count": 18, "id": "oriental-recovery", "metadata": { "id": "U5h6zCpa4sqd" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEdCAYAAAABymAfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABM7ElEQVR4nO3dd3hUVfrA8e9Jg1BDIJQMBAJIaAFCDWLFEkWE2OsW3dV1d60/jeKKq+xize7qurprd+2gLkQUNaIoKlUghB56yYQSSELLkHp+f9zcYRISGMLM3DuZ9/M8eSD33pk5dzJz33vae5TWGiGEEKErzOoCCCGEsJYEAiGECHESCIQQIsRJIBBCiBAngUAIIUKcBAIhhAhxEgiEECLESSAQIkQopcYppcZZXQ5hP0omlAnR9CmlOgBf1/x6kdZ6v5XlEfYigUCIEKCUegmYCYQDE7TWf7S4SMJGJBAIIUSIkz4CIYQIcRIIhBAixEkgELailNqmlLowQK+VpJTKUUodUkrd3cAxcUqpOUqpYqXUG0qpp5RS93r5/EuUUgN8WuiGX+u/SqmpJzkmKM5FBF6E1QUQorGUUtuA32qtv2nkUzwIfK+1TjnBMQ8DG7XWFyml4oAVQG8vn/9vwF+AqxpZPl9rSucifEhqBCKUdQfWnOSYC4GPa/7/a+ALrbXLy+efBZyvlOrSuOL5XFM6F+FDEghEwNU0/zyslFpb00zxllKqeT3H9VNKfa+UKlFKrVFKTfDY9y6QAHymlDqslHrwFB8/FzgfeLHm8X3qPDZKKXUASK55jVXApcC8Osc9q5Sa6fF7plLqW6VUpNb6KLAMuLiB92GSUmpzTdPUWqXUFfW8Tw8opVYqpQ4opaab75NSKkUptbzmsdOB496/UzmXE50HwMnORQQ5rbX8yE9Af4BtwGqgGxALzAemeuy7EIgENgF/AqKAscAhIKnO81zYwGt48/jvMZqWGipnf2CPx++FwIg6x7QHSoAhwB3AKqCtx/4XgH808PzXAPEYN2TXAUeALnXOb0nNMbHAuprXiAK2A/fVnOfVQIX5HjbmXE52Hic7F/kJ7h+pEQirvKi13qm1LgKeAG6osz8VaAU8rbUu11rPBT6v57iGnO7jwbgo5nr8HoMRTNy0MUP3eeAdjDb4cVrrAx6HHKp53HG01h9rrQu01tVa6+nARmBkncNeqDmmCPispkypGAHgea11hdb6E+Dn0zkXL87jhOcigpsEAmGVnR7/345x1+spHtipta6uc5zDy+c/3cfD8RfPYqB1PcflYDS7PKy13llnX2uMO+3jKKV+qZRaUdN0VQIMBDrUOWy3x/9LMYJbPODUWnvOBt1+wjPx7lxOdB5wgnMRwU0CgbBKN4//JwAFdfYXAN2UUmF1jnN6/H6iafHePP5kBlP74rkSqNuXkAz8B3gbuLWe5+hX5znMx3UHXgPuBNprrWMwmsuUF+XaBTiUUp7HJpzkMSc8Fy/OAxo4FxH8JBAIq/xRKdVVKRWL0Y4/vc7+xRht5g8qpSKVUucBlwPTPI7ZA/Rs4Pm9efzJ1L14fgGca/6ilHJgNNfcAfwBSK55HXN/M2AYMKee526JEcgKa469BaNG4I2FQCVwt1IqQil1Jcc3KXl9Lic7Dy/ORQQ5CQTCKh9gZMPcUvNTazKU1rocmIAxumUf8G/gl1rr9R6HPQVMrmlaeaARj2+QUqoz0A7wPP4dYJxSKlop1QbjYvoPrfUsrXUpkInR32GagDFPoW5tB631WuDvGBf1PRhNMvO9KVvNuV2JMQS0GKOjeUYjz6WtF+dxwnMRwU+SzomA88FEMMsopZ4E9mqtn/fi2MXAb7TWq/1esEZoSuciTo8EAhFwwRwIhGiKpGlICCFCnNQIhBAixEmNQAghQlxQZh/t0KGD7tGjh9XFEEKIoLJs2bJ9Wuu4utuDMhD06NGDpUuXWl0MIYQIKkqpemegS9OQEEKEOAkEQggR4mwTCJRS4TXLBn5udVmEECKU2CYQAPdg5FsXQggRQLYIBEqprsBlwOtWl0UIIUKNXUYNPY+xkHh9ud4BUErdDtwOkJBwsoy7QjQdWTlOpny2huLSCgBioiN5fMIA0lNOZWkFIRpmeY1AKTUeI/HVshMdp7V+VWs9XGs9PC7uuGGwQjRJWTlOMj7JdQcBgBJXBfdOX8HkrFUWlkw0JZYHAmAMMKEmEdk0YKxS6j1riySE9bJynNz/US4VVfWngXl/0Q6yck5lnR0h6md5INBaP6y17qq17gFcD8zVWt9scbGEsFRWjpOHZ6yi6gS5wDSQmZ0XuEKJJsvyQCCEqM2sCbgqqgDoWrKbwQX1X/ALSlyBLJpoomwVCLTW32utx1tdDiGskpXjJOPjXHdNIKqygnc/epSZ7z7ALUs/Pe74+JjoQBdRNEG2CgRChLrHZ62hovpYc9Bvf55JYvEucrv04bFvX+Oxb14hrNqoKURHhpORlmRVUUUTIoFACBspcR0bHRR/cC93LZjOV31Gc9XNz/LWiIncsuwzXs56il4t4Kkrk2UIqfAJCQRC2NQjc99Aofnr2NuoDgun3SsvwQsvcPHmJXz7+RTS4+0yDUgEOwkEQthIuxaRAIzZtoLL8ubzUuo1ONt2pF2LSOPu/667YOZMWLMGUlNhnWRlEadPAoEQNvLY5QNoQSVT5rzM9pjOvDrqKiLDFY9dPuDYQRMmwLx5HD10hJLhqQz4v08Y8/RcmVMgGk0CgRA2kp7i4KOypfQuyucvF9xOhw5tybx68HF9AVnhXfjjxfcQU3qQUTtW4Sxx8fCMVRIMRKNII6MQdlJQwMDXn4fx43njkykNHpaZnUdh/ABKI5txztblzO09EldFFZnZedKBLE6ZBAIh7CQjAyoq4PnnT3hYQYkLHRHJwoRBnLv1WJoup0wwE40gTUNC2MW8efDBB/Dgg9Cr1wkPNSeSzUscSmLxLroXFwCgQJqHxCmTQCCEDWTlOFlz8x3kt+nI2IjUk17MM9KSUMC8nsMAOGfrckDyD4nGkUAghMWycpw89f4CBuSv58PBaWwp1Sft+E1PcaCB7e3i2R7T2R0IQPIPiVMngUAIi2Vm59Fv+xoAlnXtB+Du+D0Rh7t5aBhnbl9JVGXNwjU1cxGE8JYEAiEsVlDiYphzPZUqjNzOfWptP5GMtCQiwxXzeg6lZcVRhjnXAnD4aKX0E4hTIoFACIvFx0QzzLmONZ164YpqXmv7iaSnOGgZFcHChEGUh0Vw7hZj9FBFtZZ+AnFKJBAIYbEHx/ZkyK48ljv6urd5m1n0gKuC0qholnbtz7nSTyAaSQKBEBabGL6fFhVlbOkzBIXR9u9tZlH3MNKeQ+lXuI2Oh/YD0DZa+gmE9yQQCGG1+fMB+Ouzt7P16cuYP2ms17ODM9KSiAxTzEs0hpGatYISV4Usbi+8ZnkgUEo1V0otUUrlKqXWKKUanlcvRFO0YAEkJEDXrqf80PQUB62aR7A+rgd7WsXWah6Sxe2FtywPBEAZMFZrPRgYAlyilEq1tkhCBND8+XDmmY1+eElpBSjFDz2Gcta2HPcKZjK5THjL8kCgDYdrfo2s+dEneIgQTUJWjpMrJk2D/HyeO9Kh0Xfvnv0EMUcPM3jXRvc+6TQW3rA8EAAopcKVUiuAvcAcrfXieo65XSm1VCm1tLCwMOBlFMKXsnKcPDxjFV3XGk0538T2bnQaaXN00U89hlClwmoloZNOY+ENWwQCrXWV1noI0BUYqZQaWM8xr2qth2uth8fFxQW8jEL4UmZ2Hq6KKoYWrOdIZHPWd0z0ajZxfYz5BOGURLcht8sZnLvlWD9BeWWVL4stmihbBAKT1roE+B64xNqSCOFfZpPN8Py1rIjvQ1VYeK3tp6q03Ljg/5A4lEG7NxLjOmhsr6iWDmNxUpYHAqVUnFIqpub/0cCFwHpLCyWEn8XHRNOi3EW/vVtZFt+v1vbGPh8YeYfCdTVnbVvh3icdxuJkLA8EQBfgO6XUSuBnjD6Czy0ukxB+lZGWxMi9m4jQ1Szr2h/wfjZxQ88HkNvlDEqat6rVPCQdxuJkLF+hTGu9EkixuhxCBFJ6ioM+bYuoVoqc+CQcMdFkpCU1epnJ9BQHUz5bQ3Ep/NQjhXO2LQetQalG1zJE6LBDjUCIkNR/yyrCBgxg5fPXndJs4oY8dvkAoiPDmZc4lE6Hi+hbuO20ahkidFheIxAiJFVXw8KFcN11PntKM5BMP7IbvoQxh/O548orZDF7cVJSIxDCCmvXwoEDMGaMT582PcXBh8/cBBERPHpGuAQB4RUJBEJYYcEC49/TSC3RoMhI6NUL1svgO+EdCQRCWGH+fOjY0bhg+0PfvhIIhNckEAhhhQULjNqAUv55/qQk2LQJKiv98/yiSZFAIESg7dljXKR93D9QS9++UF4O27b57zVEkyGBQIhAW7jQ+Ncf/QM15qn2APxm0ruMeXqupJkQJySBQIhAmz8foqJg6FC/PH1WjpOHVpcB0HN/Ps4SV6Mzm4rQIIFAiEBbsACGD4fmzf3y9JnZeeyOaMG+Fm3pWZQP0OjMpiI0SCAQIpDKymDpUr82C5m5hTbHdqVXTSDw3C5EXRIIhAikvDyjE3fYML+9hJlbaHP7rvTan3/cdiHqkhQTQgSSOba/X78TH3caMtKSeHjGKrbEOmjvOkiM6yBlbdo1+ZxDk7NW8eHinVRpTbhS3DCqG1PTk60uVlCQQCBEIK1fb8wd6NPHby9hppX4aaeRinpU+T4uvfKcJpduwvPCX1eV1ry3aAeABAMvSNOQEIG0fj306AHR/m2mSU9x8LdHjYR2r4xo2SSDwHuLdtQbBDx9uHhngEoU3CQQCBFI69cbk70CoUcPY5hqE0w14e0FvkprGTbrBcsDgVKqm1LqO6XUOqXUGqXUPVaXSQi/qK4ObCAID4czzjA6qJuY+moCbY4e5r4f36PzwX21tsscipOzPBAAlcD9Wut+QCrwR6VUf4vLJITv7dwJLlfgAgE02eRz4XVyNPUocjLz3fu5Z8E0/vT9m7X2yRyKk7M8EGitd2mtl9f8/xCwDmhaDZpCwLELcqADwebNxpDVJuSGUd3c/x+9PZesd+8nxnWIr/qMZvy6H+m9b0et450yh+KELA8EnpRSPTDWL15cz77blVJLlVJLCwsLA142IU5bAIaOHicpCaqqjGDQhExNT+bm1ARuXvEV705/lL2t2vPf56bxwrUZlEY15+4F0457jDQPNcw2gUAp1Qr4H3Cv1vpg3f1a61e11sO11sPj4uICX0AhTtf69RAbCx06BO41zdpHU+snqKxk6nevMzX7RSIuvYQ+m1dy/+8u5fYrR/L20PH11goen7XGosLany0CgVIqEiMIvK+1nmF1eYTwC7Oj2F9rENQnKenYazcVpaUwfjy88ALcdx/MmgVt2gDGsNnXR6TXWysocVVYUdqgYHkgUEop4A1gndb6H1aXRwi/WbcusP0DYFwgu3RpWoHg3XchOxv+8x/4xz+M0VEeilu0bbBWIOpnh5nFY4BfAKuUUitqtv1Ja/2FdUWyr6wcJ4/PWnPc3Y1SoDU4YqLJSEtqchOIgl5xsbEgTYADQVaOk27RnQj7ehF3Pj03qD8bWTlOMrPzmPT2+6S2ac/8keNJr+e4di0ieX1EOr9a/jl3L5jG3RMeBCBMGc8RrOfvT5YHAq31T0AA68rBKSvHyZTP1lBcWn/11hxWbeaeB1i6vYj3F+9w72sRGcaTVw6SL4IVzDb6AAaCrBwnD89YxZ/axjPBOQ9ncan7sxFsnwHzXFzllaTuXMVP3Yfwp5mrQanjzuWxyweQ8Ukubw8dz+8XfcILZ17Ppg4JVGuC9vz9zfJAIE7O/SWoqHJvi3EdpEfxLnoUF9CjuIDuxbtILC6gIiyCxQkDmbVlMAs6J6Ejj+W8L62o5t7pK/i/j1Zw46gEycESSBaMGMrMzsNVUcXm2K60LTtCh9IS9ql2ZGbnBd2F0DyXXvvziTtSwqKEZPf8gLrnYv4+9fAVx9UKGnpMqJNAEATML0HrsiNcm/s1v1r+OQkH9rj3V6MoaBPHtnZdaFl+lDsWfULEwo8oD4tgRXwfFiYM4vuew8lxGHej1RreW7SDrYWHef+20VadVmhZv95I99CjR8Be0lx/YEuscdHrtT+ffS3bBeW6BOY8gNE7jTv6RQnGTUxD55Ke4uC+6W2OqxWc6DGhTAJBEAjbtpVHl33GtSu/pnW5i8XdBvL20PFsi41nW0w8O2M6Ux4R6T6+ZVkpI/LXkrpzFak7VnLnwo+4Z8E03hg+kafP+zUV4cax8zcXMTlrldQMAmHdOiPdQ0TgvnLxMdE4S1xsbm9MvupZ5GRxQnLQrUvgOf4/dftKClp3YHtMF+DEayzEx0TX21cQbOcfCBIIbMTsDCsocREfE82TnQ9z7ufv8P2MmVQrxed9z+aNEems7tz7hM9T3ao1P/Qezve9hgPQuuwI//fje/xm6acMc67lzgkPkR/TGYD3F+1gePdYqSr72/r1kBzYgGuuS1DQpgOuiGb02r+T6MjwoFuXwD3+X2tSd67ih8Sh7iG4JzoX4/zLa9UKnF0Sg+78A0ECgU1k5TjJ+CSXiipN67Ij3PXhC5y78mvKW7dl269/z+2tR7EtOrbBx9cdNbR0e5E7H/uhZi2ZcuHvWNQtmcwv/8ns/95Dxrh7+brPaDRw/0e5gHSg+U15uTGz95prAvqy5t8zMzuPrbHxDDi4i6euTA66v7M5Qu6MfTvoUHqAhQnHAuqJzsXc9yo3cMuyWfxh9VeE/fHfQXf+gSCBwGJ1F9c4e+tynvnyBTodLuI/o67m3QtuZsETE7m3Tm3hZMMAzX2eo4ayk85kbaeevPjpM7w68wneHDaBp86/hYrwSBlN4SdZOU4+evdrPqiqYsrGagYHePhieorDeL0VI2HJEgjiv2+qu39gkNePMc7/CvZ8m0ry1tVcPH0Fmdl5QT2M1h8kEFjIXFwDoFVZKX/67g1uzM1mU2xXrro5kxXxSVAzUMj9hT4FU9OT3e3/N722kPmbi9gZ05lrbnqWh79/k1uXzWJowTpuufpxilu0ldEUPmaO9jpn2yYAfo7uxDSrAm7fvvDRR3D0KDRvfvLjbaRdi0iKSytI3bEKZ+s4drbt5N7ujawcJ/kRXfn9nh9pUVaKs0SGkdZl+cziUGYurjFm2wq+evOPXJ/7NS+PvJLLfv1PIwj40Pu3jebm1AQUUB4RyZQLf8cd6Q/Tb+9W/j77OZSultEUPuY55BGM0TuWpUTu29doO9y4MfCvfZoeu3wAUWGa1B2rWNQ9GZQiMlzx2OUDvHp8ZnYeSzr3IVxXM3jXBkBSU9clgSDAsnKcjHl6LomTZlNdXcV9P77H+9MnUxYRxdU3PcvT599KWWQz9/Ex0d7d9Xhjanoyz103xJ3L/aukMfx17G2M3bKU25fMIEwpEifNZszTcyVTow+YgbXX/p0UtO5AaVR0re0BFcQ5h9JTHPwnJZr2roMs6paMIyaazKsHe303X1Dict9YpRTk1douDNI0FECTs1bx/qIdaKB5xVH+Pvs5Lsubz0fJF/LoRb+vFQAAIsMUj0/w7q7HW+aXx5yg9l7KOFJ3rCJj3jssdfRnWdf+OEtc3Dd9BUu3F8nQ0tNgDt/sVZTP5tiutbYHXJ8+xr9BGAgALtizDoDMf997ynMxjL8DbGzfjaEF62ttFwapEQSI2R+ggU6H9vHRB5O4NG8BU8+/lQcvvee4IOCIiSbzGu/vek5FeoqDp6407qyUUjxy6d3kt+3Ev2Y9S4zLyACuMYaWSs2g8TLSkoiOCKPX/nw2dTDG8ls2fLNlS0hICN501N99B927N2pCXkZaEtGR4SyP72vUCLQOymG0/iQ1ggDIynHyfk2ncPKujbw+46+0LHfx26seZW7vke7jwpXihlHdAnIX7tn5nDhpNn+c+BAz3nuAv89+jt9e9ShahaGBKZ+tkQ61RkpPcdC8cDetn3CxJbar9QkBk5KCs0ZQXQ3z5hmppxvBfL/Xr00mdtUcRlUVccOVF8jn2oPUCPxsctYq7p2+Ag1ctu5HPv7gIcrDI7nq5kx3EHDERLPt6cvY/NQ4S5pi4mOiWdO5N1PH/pYLNv/MbUtmuvcVl1ZIraCRsnKczP74ewBKuveyfshi375GjaCehd/tKivHyS/+703Yv5+pRzo1+rOYnuJg0pRbAJg+SEYL1SWBwI88h4fesegTXpr1DKs69Sb9F38nL64HYKRdtbqKmpGWhALeTbmM2UljeHDe2wzNX+feL6MrTp05dDRmuzF0dHFUHA/PWGVtUO3bFw4fhoIC68pwCrJynGR8nEuv1T8D8FWHJDI+zm38e9i/v7E+w8KFPixl0yCBwE/cQUBrHvjhHSbN+y+f9juXm65/gv0tY9zH3ZSaYPndSXqKg5tSE0ApJl16N862HXlx1jO0Kz0AGAm/pFZwajyHjh6KimZvq1jrhyyaKbCDpHno8VlrqKg20krsbNuJ/LadqKjWjV9yMiwMRo2CRYt8W9AmQAKBH9z02kLeW7QDpat57NtXuXPhR3wwOI37xv+fOzmcAm5OtU8q6KnpycRER3KoWUv+MHESHY6UkPHDu+79GZ+cxp1YCDo2dDTfSPpWM2TX0iGLQTaEtMRVgdLVjNqxulZaidNacjI1FVauNGpGwk0CgY9l5TiZv7mIsOoqnv7yX9yy7DNeHz6RP6XdSXXYsSX1nrtuiG2CgOnxCQOIjgxnTefeTBucxjWr5tCtZDcAFVWaR2ausriEwcMcmtirKJ/N7S0eOup+8XiIjjbyHgWJvoXbaHf0kDvt9Ol6W3eG6mpuuO1f9Hr4CyZnyWcabBIIlFJvKqX2KqVWW12WxjInit07fQURVZX887O/cd2qOfzzzBuYOva3tRYsv9kGzUH1MYeVArw0+hqqVRh3zz+2APiR8iqpFXjp/L5xtCwrJf7QvlpzCM7vG2ddoZQyhl9u3WpdGU5BuxaRpO6ovf6Aub0xJmet4u+H2gOQUrCeKq15b9EOCQbYJBAA/wUusboQjWV2DDpLXDSrLOc/WU9y+fofeeK8W3nu7JuOCwJ2qwl4MgPUntYdeDdlHFeumUvPmhQJQOPbZ0PMd+sL6VlkBE3PGsF36wutKhJZOU4WVLdh7YLcoJg9ftmgLqTuWMX2mM4UtOkIcEqpJer6cPFODjZvxabYrqR4TCwzB3SEMlsEAq31D0CR1eVorCmfrcFVUUWzynJe+99fuWjTEiZf/AdeG3VlrePsHgRMZlqL/6ReQ1lEJPfM/9C9r8Qlw0m9UVAzoxhgU2y3WtutYN6sbGzZga4H9rrXtrbr3zIrx8mMpTsZtXM1i7oZ3xkFXDeiW6Nr02aG35z4vgx1rq81jDbUawW2CATeUErdrpRaqpRaWlho3V1VXZOzVlFcWkFUZQWvzHiCs7at4IFx9/Jeyrhax43pFRsUQQBwp7XY3zKGt4dezuXrfqBP4bZj+6VWcFLxMdH02p9PRVg429t1qbXdCuYopvw2nWhTdoQ2Rw9bP4rpBDKz8+jh3EzM0cMs7G6kndacXo3KzLG13NGX9q6DJNT0f8GxBJChKmgCgdb6Va31cK318Lg4C9tZPZgzhiOrKnjp06c4b+syHr7kTj5JvtB9jCMmmuevGxJUawOnpzi4OdVY3/WVUVdyJKo59/70gXv/aY3aCBEZaUn0Kc5nR0wXKsONCfxWpjUwayI7Y4wUzuYgALsmXisocbn7BxZ3G1hre2PdMMqomS2vWbvbM+9QVRBNsvOHoAkEdpSZnUd4VSX/mvWsuzlo+uA09/7nrxvC/EljbdkxfDJm7aUkug1vDk9n3IYFDNhzbLSJXZsU7CI9xcHoin04Oxupvx0x0ZauDmbWRMxc/l0P7K213W5iWkTSf+8WdreKZVebYzd+p1Ne8zO9sX03DkVFG81DNcJUQ48KDRIITsOeosM8//nfuWTDQqZccFut5qCY6MigDACezNEZb4yYyIFmLbnvx/fc++zapGAblZW02bGFcy4/m61PX2b5DYGZeG1nzVrV3Q7stm3itawcJwdKK0gscrLFY8RVZLg67fJGR4ZRHRZObpc+tTqMm0WE9qXQFmevlPoQWAgkKaXylVK/sbpMJ1VVxUtzXmB8zeigt4ZPdO9S4PP00VYwR2ccbN6KV0ZdxYWbf2ZITT53uzYp2MbWrVBRcWw2r8XMocHh7WI4GNWCbgf22Pbi9/isNVQDicUFbI2Nd2+PCFOnHUyPVlQDsDy+L/32biW6/Git7aHKFp8ErfUNWusuWutIrXVXrfUbVpepPuZcgZ4Pfcbno8aTtuJbnjvvV7VGBynskTbCF9JTHO4RRP8ddjn7o9vwfzW1Ars2KdiGOXvXJoHAdLRSkx/TiW4leyhxVdhy5FCJq4IY10FiXQdr1QhcPrhYm5/bHEdfInQ1g3ZvrLU9VNkiEASDyVmruG/6CpzFpfz5m1cZv+wrXjr7Rvbfc7+R1x+jHdiOM4ZPhznbuDQqmpdHXc0523IYuXM1zhJXUIxFt0JWjpMX/zMbgEu+3GOb98gcObSzbSd3H4HdRg6Z75U5B2NLrG9vqMwmshyPFcvs2kQWSLIegRc8s4g+8OO7/Hr557w64goyR9+AY30h8yeNtbiE/mPWbDKz83h36Dhu+3kmf1j4MUu6DXSPRfc8LtSZ4/Ufc26lsEUM68sibPMeuUcOte3E2dtyjHH0Stmqmc8MSolFRobUre2ONQ01dkaxJ8/P85ZYB2cWbqCvhZ34diE1gpMwE8gB/H7Rx+4Eck+ef6vtvkT+kp7iYP6ksbSPa8eHgy/hnK3Lcdj0jtJq5l13z6J8992sXd4js/kjv20nWlSU0aG0pNZ2OzC/T4nFTirCwsmvGeUENHpGcV3m5znizNEM3LGW+6blhHztVgLBCUzOWsX8zcaE518s/5yH5r1NVv9zmXzxH9xpI+z0JfK3ghIXHw26CIBrV35da7swmO9FzyJnrWYNO7xHx0YOmXMJ9tiuWcT8PiUWOdkR09k9B6NdC9+OwsvKcfJmVWdiD5fgOLDH9jOt/U0CQQOycpzumsBVq77lr3NeZk7vUTww7j53FlE7LCoTSPEx0TjbdmRez6Fcu3IO4dVV7u3CEB8TTZujh+lQeqBWR6cd3iNz5FBZ1+4AJFcWWzq3oT5mUr6eRc5azUKXDerS0EMaJTM7j8Wd+wDHJpbZpeZmBQkE9TA7hgEuyZvPs1/+k5+6D+bOiQ+571Cg6YwO8pZ5RzltcBpdDu/nvC1LASg44Ar5XC2mjLQk+h8w2rfNZHN2uutOT3Hw3tTrAPjLoJa2+/x+t74QpavpUbyrViD1dbK+ghIXGzp050hk81oTy+xQc7OCBII6zI5hDZyzZRkvzMokJz6J26+cTFlElPu4YEkg50vmHeXCvqnsbdmO63OzAaPPUdL5GtJTHDzYw/j/1liH5TOK69WqFXToYMt01AUlLroc2kd0ZRlb/di0Fh8TTVXNxDLPVBN2qLk1yOWCRx6BI0d8/tQSCDx4jg4auXM1r8x8ko0dErj16scojTr2AQmmBHK+lp7i4IgO5+PkCxm7eSmdDu1z7wv1xF2moUf3QmQk3/37VstnFDcoMRG2bbO6FMeJj4k+NmLIYzKZry/QZu12RXwf+u3dSrOKMlvV3I5TWAgXXABPPQXffOPzp5dAUMMzCCTv2sgbn0zB2SaOX1z3Vw42b+U+7ubUhKBKIOcPVVozbXAa4bqaa1fOqbU91GXlOJn32U9sat2JMX/7wb6dj4mJtqwRZKQlkXRgFwBb2hkB1B8XaLN2u6N3MpHVVZxX6rRfzc20cSOMHg05OfDxxzBx4skfc4okEHAsiyhAn8JtvPPRnymJbsPN102lqEVbwOgYfr6JTRY7HTtjOvNj9yFct3IOYTWdxqHOnEPQZfcOtrTvau+RKImJsH07VNnrb5ee4uCmdkcpjYqmsFWsX5vW0lMcPP3sbQC80qfSnkFg/nwjCBw4AHPnwlVX+eVlJBBgjCDQQPfiAt6b/ijlEZHceP0T7G7TwX1MqHUMe+PDIZfQ9eBezt62wr3Nlhe9AJny2RrKy8rpXlJguzkEx+nRw8iFVFBgdUmO06vYSYuB/dj6zHj/N6116QIJCbB4sf9eo7E+/thoDoqNhYULjYDgJxIIqOmgOljI+9MeIby6ipuum+rO0gih2TF8Io6a9to5Z4xiX4u27k5jwL53wH6WleOkuLSCrgf20KyqstY6xbYciZKYaPxrw34CNmyAPn0C93qjRtkrEGgNmZlw7bUwfDgsWAC9e/v1JSUQAAPCXbw3fTJtjh7hl9f+hU0dEtz7JAgcz+xoqwiP5JOBF3DhpsXEHS4GbHwH7Gfmqm3udYptNoegrjmuFgDc/8xMe82qLSszglOgA8G2bbB3b+Besx5mUsunx94KDz6I86LxRsdwhw4nf/BpCslAYL7hiZNmM+7Rmbz+3sN0ObSPW655nDWdjcirkCDQELOjDWD64DQiq6u4evWxkQy2vAP2M3PVtp77jXWKt7Q/1pxht5EoWTlO7l9sBO6uJTabVbtlC1RXBz4QgKW1ArN/qdOa5Tww7x1mJ43h/GG/I+XZH0icNNvvwTrkAoH5hjtLXLRxHSLz1QeIKdjBtMkvsnvgsCabRdTX0lOMMfJbYx0sTEjm+txslDbSBMf4IDlYsOpV5KQoug0l0W3c2+zWt5SZncdBHc7uVrF0O7AHsFFNbsMG499ABoKhQyE8POCBwPOG9P6Pcok8dIAXZj1LQZs4Jl16N+VaUVxagQa/B+uQCwRTPluDq6KK1mVHeOejP9N7/w5uv+IR3ojqyfxJY22xmlSwyEhLIjJc8eHgNLqX7Gb09pUAHD5aaY+7ywAylzr0TDbnud1OjmUh7UzXmkDgud0qWTlOXnrZSN998exdgfsMtWgBgwYFNBBMzlrFvdNX4CxxoYGq6mqe+fIFOh0u4q4JD3KoWcvjHuPPYB1SgcDs0GtZVsp/P3qMfnu38vv0P/FDz2GWfwmCUXqKg5ZREWT3OZPi5q25sabTuKJa2+PuMoCqa6ZQ1E02V23DqRXu9YtjOtUKBFb2ZWTlOMn4JJf2BdspbBnDhqPhZHySG7hgMGoULFliNEv5mWceM9PNK77k0g0LePacX5Eb33BTor+uU7YIBEqpS5RSeUqpTUqpSf56nczsPKLLj/LmJ1MYvGsDd018kLm9RwL27NALBgdcFZRFRJE14Dwu2riIVmWlgPV3l4HmiImmddkROh4pducYMrfbjTsLadtOdDm0n4iqSstn1U75bA0VVdpYp7hmIllFlWbKZ2sC8vrLO/eBgwe58PZX/d4eX/cmqd/eLTz67Wt813MYr49MP+Fj/XWdOmkgUEp9o5Qa7JdXN54/HHgJuBToD9yglOrvj9faX1jC6zP+wnDnOu69/AGy+5zp3me3Dr1gYX4wP+97Ns2qKrhg0+Ja20NFRloS/cwZsbH2SzbnyezsP9KlK+G6mhR1yPJZtcWlNZ3txc5aOYbM7f6UlePk0d1GU0xKwXq/t8c7PW6SosuP8uKnz1IS3ZoHxt2HVmEoICY6ksjw2u2K/vw8eVMjeBB4Tin1llLKt7lgDSOBTVrrLVrrcmAa4Ps51MA/vn+F0dtX8cC4e/m83znu7THRvs11HkrMu8vljr7satWey/Lm2/YC6E/pKQ4eqhmav8WuyeY8pKc4+NMfxwHwcVoXS8tpXnBblx0h7khJrRxDgZCZncfaNl042KwlQ3YZd+v+ao+vm5jxL3NeJrHIyb3j72d/yxjatYhk69OXseKxi8m8enCtZXD9+Xk66VKVWuvlwFil1FXAV0qpGcCzWmtf1f0dgGe2snxgVN2DlFK3A7cDJCQk1N3tlcg/T+ahtwYzs++57m3RkeE8PsE3Kx+FIs+l/75MGsNNuV+SeXEPxtv0AuhPw8oKITycb1+6FaKiTv4Aq5mTyizOOWRecHvUJJvzTD8dE+3/EWgFJS60CiO38xkMKdjg3u70cfOmZyobgAlr53HN6m/455nXs7D7YMLDVK1V2NJTHAEL0F71ESilFJAH/Ae4C9iolPqFj8pQ37iK47rYtNavaq2Ha62Hx8XFNeqFLpp4NmMeuydgUTZUmEv/3frPB2lWWcH4ncutLpI18vKgZ8/gCALAp4WKyrAwXnpzjqWTyo6t6lYzB6Pdse9jIG7SzGbMFfFJJBVuo3nFUcC4MPnyPTFT2QC0KHcxee7r5HRJ4oUxNwDw92sGW3Yt8qaP4CfACTyHcff+a+A8YKRS6lUflCEf6Obxe1fAbwlQzIuWDBP1g9GjweEwcqSEovXroW9fq0vhlawcJ5NmrWNX6zjLl2o0L8Q9iwqoUmHu9C6BarLNSEtCASvi+xChq0nevQkw7kZ92TzkWcO4bclMOh4p5i8X3EZVWDiOmGhLr0Xe1AjuABxa64u01o9qrT/XWm/SWt8FnO2DMvwMnKGUSlRKRQHXA7N88Lwi0MLCjOyIX34Jhw5ZXZrAqqoy0gUnBUffSGZ2Hq6KKna27US3EmsnlWWkJREZpkgsdpLftiPlEZFEhqmANdmmpzjQwIouxt/Os3nIV6PfsnKc7qaPuMPF3L5kBrOTxpDj6GuLJW9PGgi01qu1bjDR/GWnWwCtdSVwJ5ANrAM+0loHZsyY8L1rrjHyxXz2mdUlCawdO4zzDpJAcGxSWSf37GLP7QGnjAXrt5rNQgGeiOeIiWZ/yxh2tO3k7jAG341+82wWumf+B0RVVZB5zi8Be2Q2Pq15BFrrLb4ohNb6C611H611L631E754TmGRM8/EFdeJeU/8OyA5Umwjr+biESSBwHNSWccjxTSrKAOgbQA6Z+vKzM6jorKaxOJj6bsrqgI7KfHYimVJ7hqBAs7v27j+yLrMANtr/06uz83mvZRxbKs5VzuksrHFhDLRdGTl7uLjHqmk5i2hRVmpvRKa+UlWjpPnXzJqQJdl7w2KczWbY/LbdgKg60Ej8+aR8sCnBykocdHxcBGtyl21ZmUHsnaSnuLgqmEOVnRJwnGokLjDRWjgf8ucPnk/zMD70Ly3KY1szr/OvB6wz4RDCQTCpzKz85jV50xjctnmnwEbJTTzAzOJYYf8rRxo1pI1Fc2CIvClpzho1TyCnTWBwOwnCPSdOBgXyZ7FxvvlOZks0JMSv1tfyIp4I9ldSoFv5xNkpCVx1q51XLxxES+nXk1Ri7a2mm8jgUD4VEGJi2WOfuxuFcu4vJ9qbW+KzE7XnkVOI7WEUkET+EpKK9yBwMrkcxlpSfSpmZVtTiaz4iJZUOJiTadelIdF1Oon8MX7kT4knheWv8/eNh14a/gE2w1dP+mEMiFORXxMNM4SF18mjeGG3GxalpVypFmLJptywnMM/E89Uo7bbmdtoyMp1O0oC4+s1WEc6H6C9BQHA9qXUxYRxe7WHXDERJORlhTwi6Tx2YV1HRPdNQJze2Nl5TjJzM5jyKI5vLR6Bcsf/wfrHvPPusOnQ2oEwqfMTrcvksbQvLLc3TxUakHbcyDEx0TTsqyUzoeLarVvB0PgUwq0CiO/bcdaNQJlQersM0oKaNYviS3PXG7Z/J5jHcZ9SN69ibDqqtOqmZjNhnv3HyRj3tusi+vBL8r72PJ7IIFA+JSZ0GxT78Hs8WgeKi6tCIq281OVkZZE30O7gWPLU9qp7fdESmoSuuXXGUJaEoBEbyZzcZYt85czV7ez9PNhfna390qmVbmLMeV7T6v5xmw2vCnnS3qU7OLp827hSJVvJ6n5igQC4XPpKQ5aREfxRdIYztuyjBblRjNJsLSdn4r0FAcPmyl7giDZnCf3EFKPSWWe2/3NXINgz/5DdCvZzdrWnQO7BkE90lMc/HnqLQC8O7Dxq8tl5TiNVRCPHubuBdP4qftg5iUOBezZbCiBQPiFs8R1rHlo05Ja25uSrBwnq+YuoUqFUZ7Y05K27cZyr0sQ04l2Rw/Rqqw0oLUZcw2Crgf2EFldxdZ2joCuQdCgM86gtFUbpr/4CT0mzabXw18clzX0RMwmIYA/LPqYGNchnjz/N+42Nzs2G0ogEH6hFCzt2r+meWh+re1NhfmFjyvYzs62ndh2uCqomr/MppDSLkaqr6HVJQGtzZhrDSQWG6nFzBFDgViD4EQmf7qan+N6M6imw7hKa95btMPrYGA2CTkO7OWWpbOYMXAsazv1BOzbbCiBQPiF1kZH5Jd9zuT8LUvdzUMNJisJQseGjh5bpzjYmr/SUxz89b7LAXhnbEdLajM9i4zA6dnZbqUPF+9kmaMfSYXbSSjeVWu7N8xa7wM/vINWir+dfSxRs12bDSUQCL/K7nMmzSvLOWvbCquL4nMFJS6UriaxqMCyGbE+YdG6BOZaA4lFToqi21AS3abWdqtUac20QRdTHhHJXQum19rujXClSN61kSvWfs/rI9LZ3aaDe7sdgwBIIBB+0q6F8WX+uWt/DjZrydiaYaTm9qYgPiaa+IP7iK4sY4vHOsV2bAM+ofbtoVUr2LYtoC/7+IQBhGHUCLa1MxY/DCMwaxCcSLhS7G3dnveGXMqVa+bSo6bGEu5lu2ZVdTV/+v5N9rVoy8ujrj623cbVYQkEwi8eu3wAkeGKyvAI5iUOZezmn4kK07VWYAp2GWlJ9D1grqpl3OnZtQ34hJQyagWbNwf8pcPDFb2K8tnUvpv7d6vdMMooy8upV1MeHsndC6bV2n4iWTlOLtr8M6N3rOL5MTdyuFkL9z675BWqjwQC4RfpKQ73mqtze4+k45FiXu1v36pxY6SnOLinWzUAm2O7BdXQ0eMkJRkL6wRQZnYezUsP0+lwkZGeA2tyHdU1NT2Zm1MTKG4Vy9tDL2Pi2nnc56g8aZbQrBwnkz9ZwUPfvcnmWAfTBqe599n9BkFSTAi/ca+5um8QdHqO8zYuwQdLWNjKoANOiInh53/dFNRDota368oZm2fQ/4GZdOjQNiDDYAtKXAzebyxPuTm2W63tVpuanmxc+AtHQGI29yyYBndNPOFjHp6xkiuXfkHvonxuu3IyleHG5TVcKdvfIEiNQPhfhw7sHzSM9a9/2PTWKMjNhcGDgzoIZOU4eW1fM8J1NT2KCwKWOjw+Jpre+42ROJvs2scSFwd33w3Tp8Pq1Q0eNjlrFeGHD3PvTx+wuNtA5vQe5d5XrbWtgwBYHAiUUtcopdYopaqVUsOtLIvwn6wcJ/+NHUjfgo10PLSv6axRUF0Nq1YZgSCIZWbnsTbGuFD13mdcmAMxDDYjLYmkEidl4RHudYrt2IQy+6IbORIVzRdX39HgTcwHi3fwu8X/I660hCfOv7XWjYGtAlsDrK4RrAauBH6wuBzCjzKz8/iqhxHnx25eCgTfePt6bd4MR44EfSAoKHGxpZ2DapT7Dt3c7k/pKQ7GRx7A2aEr1TULuNutCSUrx8kDc/N5fdgExuXNp/WGtcfdxDz6vxX8esmn3L5kBp/2O5eVXfrUeg67Bbb6WBoItNbrtNZBfjUQJ+MscbGxQwI723Zi7OYmlG4iN9f4N8gDQXxMNGWRzdgZ06lWIAjEnWyXXdvoedYwtj59mWVZR0/EnDT4xoh0DjZryb3zP6h1E/PtJ3O58q7r+fPc1/ipxxCmXHh7rceHqcbnKwokq2sEXlNK3a6UWqqUWlpYWGh1ccQpCFcKlOKb3iM5a1uue31cb8dl21ZuLoSHw4DgHhJr5hza1L6bOxAEpImmvNyoVfXr59/XOQ1mrehg81a8MXwil2xYyIDdmyjcf5B1v3+As69Lo3vJLu6+/AF+c9WfKWrRttbjbxyVYEWxT5nfRw0ppb4BOtez6xGt9afePo/W+lXgVYDhw4fbd2aGOI45kWZurxHcsuwzztyxku96jbD1BBuv5OYawy6bN7e6JKfFvGPdMy+Rs7atoFubKO6/tL//72Q3bYKqKujb17+vcxrMhZYA3hwxkVuXfsqUb16hVVkpffdtJ6v/ufzlgtuPCwAA0ZFhtliY3ht+rxForS/UWg+s58frICCCmzmRZnG3ZA5HRbuzkdp5go1XzBFDTUB6ioMbf51Gs6oKfrw2MTDNGea8BRvXCMzaEsChZi15deSVDHeuo+3Rw9x61Z+59/KMeoOAAp66clCAS9t4QdM0JIKX+WUqj4jkxx4pjN38M9ERYUHRidag4mLYsaPJBALg2AV53Tq/v1RWjpNXXpkNwAWf77LtCDIzQ6t50/LayCu577L/46Lf/oe5vUfW+xgF3JSaEBR9Ayarh49eoZTKB0YDs5VS2VaWR/iH55dpbq8RxB/ax78HhgfVF+U4K1ca/0ogOGXu9N3OreS3iWNzKbYeTpye4mD+pLEooDwikpkDx9ZKHeEpXCmeu25I0DQJmaweNTRTa91Va91Ma91Ja5128keJYGR+mTLfehiU4nyPxWqCUhMZMVRLTAx07uz3QGCOxOm9fydbapb3DIbhxN6Movr7tYOD8gZHmoZEYHXqRNGAIax55f3gnmWcm2vMOu1c3ziIINavn98DQUGJC7Sm1/5jyebc223Ms7+gPmN6xQZlEAAJBCLAsnKcvNs+mQH562l/pDh4Zxk3gdQS9TIDgR9HdMXHRNPl0D5aVhx1J5szt9tZ3f4CU7hS3JyawPu3jbaoZKdPks6JgMrMzqNt4nDumfcO52/+mY8HXexuFgiau6nKSiPvzJ13Wl0S3+vfHw4ehF27ID7eLy+RkZbEZ39bBOAOBHZMLVEfdyLFJkZqBCKgCkpcrO2YSEHrDlxQs1iNuT1obNgAZWVNq3/AFIAO4/QUB/c4KgHY3D7I03c3EVIjEAFlTtCZ22sEV6z5jqjKCsojIm3fLFBLU+woNnkGggsu8NvLDDq820jf/UJwp+9uKqRGIALK7HD7tvdIWlYcJXXHyqBpFnDLzYXISFvPiG2srF1VHGreknff/NK/Hfnr1hlBR4KALUggEAFldrhtTx7J0Ygozt+63N1HEDQdxrm5Rlt6VJTVJfGprBwnD89czcbYrvTev9O/Hfnr1zfJQBqsJBCIgEtPcXD35YNZlDCIc2v6CZwlLjI+zrV1MMjKcTLm6bns+WkJX4Z1tHVZG8Mc3++ZfM4v4/tLSmD3blunlgg1EgiEJR6ftYa5PYfRs7iAHkXGBbWiWvP4rDUWl6x+5mxYV8FuOh0uYmlMQnAOez0Bs8N+U/tuxB0poc3Rw7W2+4yZY0hqBLYhgUBYosRVwXe9RgBw3pZltbbbkXm33G/vVgDWdUwMitmwp8LssDcneZmrlfm8I98ckSQ1AtuQQCAsszOmM5tju3L+lqVWF+WkzLvifnu3AEYg8NzeFHiuSwDQe/9On3fkZ+U4ee+/2ZSFR3DO9K1NqkYVzCQQCEuYg0W+6zmM1B2riC4/Wmu73Zh3xf32bmV3q1iKa1IPB9Ww15MwO/J19+6UhUcy5Mgun47vN5vXOhVsY2s7BzsOlTe55rVgJYFAWMLMYPBdrxE0q6pg9I6Vtbbbzfl94wDov3eruzbgub2pSE9x8OOfLqLZgH7c2PqITyd5PT5rDa6KKnrt3+meUdzUmteClQQCYQkzX8vPXQdwJLK5u3nIrovVzF65i8iqCnrtz68VCGav3GVhqfzIx8nnsnKclLgqiKqsIKFkd1AlmwsFEgiEJTwXq5nfYwjnb15q68Vqiksr6L1/J1HVlayLS6y1vSlaF+Ogets2+t4/wycTy8y7/u7FBUTo6qBKNhcKJBAIS3hmcvy+53C6HtzLiynNbZ1vxhwxtLZjT4tL4l9ZOU5eKWxOmNb0LHL6ZGKZedffqygfgM2xxwKBXYN/KJFAICxjLlbz5GsPAnDB1mUneYR1YqIj6bd3K0cjotgWG19re1OTmZ3H2hjjHHvv3wGcflu+eddvTlQzF6Rp1yLS1sE/VFi9VGWmUmq9UmqlUmqmUirGyvIIi3TrBsnJ8MUXVpekQeMHd6Hf3q3kdehOVZixOElkmOLxCQMsLpnvFZS42NbOQZUKc88lMLc3ltkU2Gt/Pvlt4nBFNSc6MpzHLm96718wsrpGMAcYqLUeBGwAHra4PMIq48bBjz8aufBtJivHyf+W5tPPY8SQAq4b2a1J3s3Gx0RTHhHJ9pjO7jt4c3tjmU2B/Uqcknrahqxes/hrrXVlza+LgK4nOl40XT/2HgGVldzxq2dst3xlZnYerYsLae866A4EGvhufaG1BfMT8+59c/tu9N5vtOn7YmJZ+uAu9C1xcu6Es5k/aawEARuxukbg6Vbgy4Z2KqVuV0otVUotLSxsml/AUJWV4+QPW5pxsFlLztuy1HbLVxaUuOjvkVrCc3tTZN6973Yk0qO4gITWUb65e8/Ph9JSSS1hQ34PBEqpb5RSq+v5mehxzCNAJfB+Q8+jtX5Vaz1caz08Lq5pTeIJdZnZeRyqVvzQI8WYT6A1rooqpnxmjwR08THR9Cs0AsH6uB61tjdV6SkOfnHLJURVV/LD1Qm+uXuXZHO25fdAoLW+UGs9sJ6fTwGUUr8CxgM3aW3XeaXCn8w76+97DafT4SL33XdxaYUtagUZaUkMLNxGfpuOHGzeCgieNXZPi6+XrZRkc7Zl9aihS4CHgAla61IryyKsY95Zz0scBsB5Hkno7FArSB8Sz/lFG9mQ0BcFodPRad65+yoQrF8P7dqB1Ohtx+o+gheB1sAcpdQKpdTLFpdHWMC8sy5s1Y6VnXtz/uZjgcAWM3e3bqXF7gLG/v46tj59Weh0dLZpAw7HaQcCc0GfRbN/YmWbeLJWFPiogMJXrB411Ftr3U1rPaTm5w4ryyOs4XlR/a7ncIYWrKet65CFJarj+++Nf887z8pSWOM0cw6ZGUedxaVGnqY28bYaCCAMVtcIhACOzdD9vudwwnU1525dXmu7pb7/3mjOCMW27aFDjTWaDxxo1MPNBX367NtOXGkJK+KTJOOoDUkgELbw+IQBRIYpcrucwb4Wbblg0xJ7zNzV2ggE551n38US/GniRKioaPSsb3MgwMUbFwHwTe+RtbYLe5BAIGwhPcVB5jWD6RLbijm9RzF28xJUeTmZ2XnWNiNs2wY7d4ZmsxBAaip07gwzZjTq4eZAgIs2LmZ5fBKFrWJrbRf2IIFA2EZ6ioOMtCS+638WrctdnLl9hfWTy0K5fwAgLAyuuMKoEbhO/S4+Iy2JxNIiBu/eyNdnjAZCZOhtkJFAIGwlMzuP7x3JHIxqwSUbFgLWrWKVlePky399yL4WbRnz6a7Q7eC84gpjRvDXX5/yQ9NTHDzfwshgOueMUaEz9DbISCAQtlJQ4qI8IpK5vUdw0cZFhFVXubcHUlaOk4f/t5LkzStY1C0Z54GjoTva5bzzICYGZs5s1MMHL58HSUl8+9odoTP0NshIIBC2YrYdZ58xmvaug4zIX1tre6BkZufRfl8BXQ8WsighGQjh9XUjI2HCBJg1y+g49lJWjpO0x2ZR8e1c3us0JDSDaJCQQCBsxcx8Oa/nMI5GRHHJhgWWtCkXlLgYvWMVgDsQmNtD0pVXQnExzJvn1eHm/IGknJ+IrK5iRsLw0K1RBQEJBMJWzMyX7TrG8kPiUMZtWsRTVwwMeHNCfEw0qTtXsa9F21oLrYfqaJdZHQfgimzOuw8+51WacHP+wMUbF1HYMoYcmT9gaxIIhO2YS1hePPn3dDpQSHrVroCXIePiPozesZrF3Qa65w+E6miXrBwnD32xibk9h3HxxkUUFB856d19QYmLqMoKzt2ylDm9R6FVmHu7sB8JBMK+xo+HiIhGj2E/HentKog/uJf1SUNDK9FcPcy7++w+o+l0uIiUgryT3t3Hx0QzesdKWpe7+PqM1Frbhf1EWF0AIRoUGwvnn28EgiefDOzM3pr5A/c/+TvuHxDa6+qad/Hf9RpBeVgEaRsWstzR74R39xlpSRz9+O8ciWzOwu6DgdCtUQUDqREIW1sxYixs2MBFt70csCUszfkD+6PbMGbW7pDv4DTv4g81a8n8HoO5ZMMC0JrmkQ1fPtIHd2HijmUsThpJeURUSNeogoEEAmFbWTlO7j6SQDWKtA0LAjLLuNb8gYQQnz9QIyMtyX2h+KrPmXQv2U2/wq24KqqZnLWq/gctXUr0vj2Mfei20ErdHaQkEAjbyszOY0fztix39A3YLGOZP3C89BQH1LTKfdN7FFUqjEvyjL/Hh4t31v+grCwID4fLLgtMIcVpkUAgbMtsg/6qz2gG7tlM15Ldtbb76zXd8we6yfwBU3XNIrL7W8awtGt/0jYsAKCqodVls7Lg3HONFcmE7Vm9VOVflVIra1Yn+1opFW9leYS9uGcZ9zkTgLSaWkGYUn5rqjHnD+yPbsPGDgnHlSVUhXt01H/VZzR9922nR5Gz1na3DRuMxWzS0wNXQHFarK4RZGqtB2mthwCfA3+2uDzCRsxZxjtjOrOmY09381CV1n5rtzfnDyxKSA75+QOebhh1bFJddh8ji2jaxoVERdQTlD/91Ph34sRAFU+cJquXqjzo8WtLoIF6pghF5izjcKX4qs9ohjnXEXe4CPBPu31WjpP3ps0j/uBeliQMAkJ7/oCnqenJ3JyaQJiCgjYdye18BpfmzSfy0EEmf7LCHQwmZ61i6Qv/ZXWnXvT6z+qGO5OFrVhdI0Ap9YRSaidwEyeoESilbldKLVVKLS0sLAxcAYWl0lMcVGnNV33OJAztXukKwOnDdvusHCcZn+SSuPpnABZ0G0hkuCIjLSnkg4BpanoyXdqazXWjGbJrIyv/eT2rnxzPJaN6cTimPb+9JY2h+ev4+oxUqrTmvUU7JBgEAb9PKFNKfQN0rmfXI1rrT7XWjwCPKKUeBu4EHqvvebTWrwKvAgwfPlxqDiEkXCk2dkhgc6yDCet+4P2Uce59WTlOn1yop3y2hooqXbt/oEoz5bM1Egg8mJ3mbw2bwN5WsbQ9epiW5S5alrtoVV5KqzIXqzv35qPki9yP+XDxTqamJzf0lMIG/B4ItNYXennoB8BsGggEInRVaQ1KMW1QGo98/yb992xhbaeegDHc0xcX6uLSCpSuZvT2lbXyCxWXep92ORTEx0TjLHHhimrOJ8nefbUbHFkkbMPqUUNnePw6AVhvVVmEfTlqRuxMH3wxRyKb85ulWe59vhzWee6WZcQf2scXfc/y2XM2NWYH/qmod2SRsBWr+wieVkqtVkqtBC4G7rG4PMKGMtKSUMDB5q34aNBFXL72B3ensS+GdZodnb9e9jl7WsXyVc1wVYCY6MjTfv6mxLMD31ueI46EPVk9augqrfXAmiGkl2utQ3cev2hQeoqDm1ITUBht0xHVVfxy+WyfDevMzM6jR5GT87Yu4/0hl1IZfqzF9PEJoZ1wrj7pKQ6qvWzuaREZJv0DQcDqGoEQXpmansxz1w2hKrEnc84YxS9XfMGzl/b2Sf9AQYmLXy6fTXlYBB8OvqTWPukorp83NbHIMMWTVw4KQGnE6ZJAIIKGuWBN2hvP0tZ1iMtXfeuT5+0VDVev+oYv+o6hsNWxlAiOEJ9NfCIZaUlEhjXcPBQTHUnmNYMlkAYJWY9ABJ+zzoJhw+C55+C22yDs9O5n/la2kjblpbwzdLx7m8wmPjHzAv/4rDWUuIyRVe1aRPLY5QPk4h+EJBCI4KMU3Hcf3HwzfPUVjBt38sc0RGuGfPoexf0Gsad/CurAUeJjomUimRfSUxzyHjURSgfhGN/hw4frpUuXWl0MYaXycujZE/r2hW++afzzzJ0LF1wAb70Fv/61z4onhB0ppZZprYfX3S59BCI4RUXBnXfCt9/CypWNf54XX4T27eH6631XNiGCjAQCEbxuv53yZs35+OYH6DFpNr0e/uLU8tps325kyrztNmje3H/lFMLmJBCIoDX5ByfT+l/AhLXfEXe4+NSTnL38svHv73/vv0IKEQQkEIig9eHinbw1fALNqiq5OeeLWttPyuWC114zcuYnJJz8eCGaMAkEImhVac3WWAdzeo/klqWfMqQgz739pKZPh/37jX4GIUKcBAIRtMx8N1Mu/B3FLdrw3vTJpO4wOo4bWr0sK8fJmKe+ZdWkqWzt2J2smD4BK68QdiWBQAQtM5lZfttOXHPjMxS0juPtjx5j7KYl9S5lmZXj5IW3vuWJ1x4iec9mXk25nIdnrvbb+sdCBAsJBCJomcsnAuxt3Z7rbnyKvLjuvDLzCS5Y+R33Tl/BmKfnGhf66mo2P/4Ms165g5H5q5lywW1MG3yxX5a8FCLYSCAQQW1qejJmxpviFm258fonWR7flxdmZXJdbjbOEhcv/PtzlnYfxP2z/sXy+L5cfOtLvDV8IloZH39frmkgRDCSFBMi6JmrZgEcbtaCX107hZdnPsUzX/2Ls7at4OKNizgaEcUD4+7lk4EXuFcf83y8EKFMagQi6NVdNetoZHNuu2oys5PGcPn6H/m21wgu/O3LfJJ8IapOEJDkckJIjUA0AWbis8zsPHfNoCI8krsmPMg/igvY3P7YClkaI710QYlLkssJUcMWgUAp9QCQCcRprfdZXR4RfMxMmFk5Tu6dvgKA6rDwWkEAjCAwf9JYC0oohH1Z3jSklOoGXATssLosIvilpzgY0yu23n1hCmkGEqIelgcC4DngQYxauxCn7f3bRnNzakKtPuEWkWH849oh0gwkRD0sXY9AKTUBuEBrfY9SahswvKGmIaXU7cDtAAkJCcO2b98euIIKIUQT0NB6BH7vI1BKfQN0rmfXI8CfgIu9eR6t9avAq2AsTOOzAgohRIjzeyDQWl9Y33alVDKQCOTWDOnrCixXSo3UWu/2d7mEEEIYLBs1pLVeBXQ0fz9Z05AQQgj/sENnsRBCCAvZYh4BgNa6h9VlEEKIUGTpqKHGUkoVAo0dNtQBCLXmJznn0CDnHBpO55y7a63j6m4MykBwOpRSS+sbPtWUyTmHBjnn0OCPc5Y+AiGECHESCIQQIsSFYiB41eoCWEDOOTTIOYcGn59zyPURCCGEqC0UawRCCCE8SCAQQogQ12QDgVLqEqVUnlJqk1JqUj37lVLqhZr9K5VSQ60opy95cc431ZzrSqXUAqXUYCvK6UsnO2eP40YopaqUUlcHsny+5s35KqXOU0qtUEqtUUrNC3QZfc2Lz3VbpdRnSqncmnO+xYpy+pJS6k2l1F6l1OoG9vv2+qW1bnI/QDiwGegJRAG5QP86x4wDvgQUkAostrrcATjnM4F2Nf+/NBTO2eO4ucAXwNVWl9vPf+MYYC2QUPN7R6vLHYBz/hPwTM3/44AiIMrqsp/meZ8DDAVWN7Dfp9evplojGAls0lpv0VqXA9OAiXWOmQi8ow2LgBilVJdAF9SHTnrOWusFWuviml8XYWR8DWbe/J0B7gL+B+wNZOH8wJvzvRGYobXeAaC1DoVz1kBrZaQxboURCCoDW0zf0lr/gHEeDfHp9aupBgIHsNPj9/yabad6TDA51fP5DcYdRTA76TkrpRzAFcDLASyXv3jzN+4DtFNKfa+UWqaU+mXASucf3pzzi0A/oABYBdyjta4OTPEs49Prl22SzvmYqmdb3XGy3hwTTLw+H6XU+RiB4Cy/lsj/vDnn54GHtNZVStV3eFDx5nwjgGHABUA0sFAptUhrvcHfhfMTb845DVgBjAV6AXOUUj9qrQ/6uWxW8un1q6kGgnygm8fvXTHuFk71mGDi1fkopQYBrwOXaq33B6hs/uLNOQ8HptUEgQ7AOKVUpdY6KyAl9C1vP9f7tNZHgCNKqR+AwUCwBgJvzvkW4GltNJ5vUkptBfoCSwJTREv49PrVVJuGfgbOUEolKqWigOuBWXWOmQX8sqb3PRU4oLXeFeiC+tBJz1kplQDMAH4RxHeInk56zlrrRK11D22kOf8E+EOQBgHw7nP9KXC2UipCKdUCGAWsC3A5fcmbc96BUQNCKdUJSAK2BLSUgefT61eTrBForSuVUncC2RijDt7UWq9RSt1Rs/9ljBEk44BNQCnGXUXQ8vKc/wy0B/5dc4dcqYM4c6OX59xkeHO+Wut1SqmvgJVANfC61rreIYjBwMu/8V+B/yqlVmE0mTykg3ylQ6XUh8B5QAelVD7wGBAJ/rl+SYoJIYQIcU21aUgIIYSXJBAIIUSIk0AghBAhTgKBEEKEOAkEQggR4iQQCCFEiJNAIIQQIU4CgRA+oJT6Til1Uc3/pyqlXrC6TEJ4q0nOLBbCAo8Bf1FKdQRSgAkWl0cIr8nMYiF8pGY1sFbAeVrrQ1aXRwhvSdOQED6glEoGugBlEgREsJFAIMRpqlkZ6n2MVaOOKKXSLC6SEKdEAoEQp6Em1fMM4H6t9TqMTJiPW1ooIU6R9BEIIUSIkxqBEEKEOAkEQggR4iQQCCFEiJNAIIQQIU4CgRBChDgJBEIIEeIkEAghRIj7f1xIQV9masW6AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "X_grid = torch.from_numpy(np.linspace(0,1,50)).float().view(-1, d)\n", "y_hat = neural_network(X_grid)\n", "plt.scatter(X.numpy(), y.numpy())\n", "plt.plot(X_grid.detach().numpy(), y_hat.detach().numpy(), 'r')\n", "plt.title('plot of $f(x)$ and $\\hat{f}(x)$')\n", "plt.xlabel('$x$')\n", "plt.ylabel('$y$')\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "active-control", "metadata": { "id": "QU2jE3Ev4sqf" }, "source": [ "## Briefer Sidenote: Learning rate schedulers\n", "\n", "Often we do not want to use a fixed learning rate throughout all training. PyTorch offers learning rate schedulers to change the learning rate over time. Common strategies include multiplying the lr by a constant every epoch (e.g. 0.9) and halving the learning rate when the training loss flattens out.\n", "\n", "See the [learning rate scheduler docs](https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate) for usage and examples" ] }, { "cell_type": "markdown", "id": "gothic-cannon", "metadata": { "id": "VQkwhm5k4sqe" }, "source": [ "## CrossEntropyLoss\n", "So far, we have been considering regression tasks and have used the [MSELoss](https://pytorch.org/docs/stable/nn.html#torch.nn.MSELoss) module. For the homework, we will be performing a classification task and will use the cross entropy loss.\n", "\n", "PyTorch implements a version of the cross entropy loss in one module called [CrossEntropyLoss](https://pytorch.org/docs/stable/nn.html#torch.nn.CrossEntropyLoss). Its usage is slightly different than MSE, so we will break it down here. \n", "\n", "- input: The first parameter to CrossEntropyLoss is the output of our network. It expects a *real valued* tensor of dimensions $(N,C)$ where $N$ is the minibatch size and $C$ is the number of classes. In our case $N=3$ and $C=2$. The values along the second dimension correspond to raw unnormalized scores for each class. The CrossEntropyLoss module does the softmax calculation for us, so we do not need to apply our own softmax to the output of our neural network.\n", "- output: The second parameter to CrossEntropyLoss is the true label. It expects an *integer valued* tensor of dimension $(N)$. The integer at each element corresponds to the correct class. In our case, the \"correct\" class labels are class 0, class 1, and class 1.\n", "\n", "Try out the loss function on three toy predictions. The true class labels are $y=[1,1,0]$. The first two examples correspond to predictions that are \"correct\" in that they have higher raw scores for the correct class. The second example is \"more confident\" in the prediction, leading to a smaller loss. The last two examples are incorrect predictions with lower and higher confidence respectively." ] }, { "cell_type": "code", "execution_count": 19, "id": "eastern-joint", "metadata": { "id": "L2YrNsXI4sqe" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tensor(0.1269)\n" ] } ], "source": [ "loss = nn.CrossEntropyLoss()\n", "\n", "input = torch.tensor([[-1., 1],[-1, 1],[1, -1]]) # raw scores correspond to the correct class\n", "# input = torch.tensor([[-3., 3],[-3, 3],[3, -3]]) # raw scores correspond to the correct class with higher confidence\n", "# input = torch.tensor([[1., -1],[1, -1],[-1, 1]]) # raw scores correspond to the incorrect class\n", "# input = torch.tensor([[3., -3],[3, -3],[-3, 3]]) # raw scores correspond to the incorrect class with incorrectly placed confidence\n", "\n", "target = torch.tensor([1, 1, 0])\n", "output = loss(input, target)\n", "print(output)\n" ] }, { "cell_type": "markdown", "id": "protecting-drain", "metadata": { "id": "5e0topiJ4sqf" }, "source": [ "## Convolutions\n", "When working with images, we often want to use convolutions to extract features using convolutions. PyTorch implements this for us in the `torch.nn.Conv2d` module. It expects the input to have a specific dimension $(N, C_{in}, H_{in}, W_{in})$ where $N$ is batch size, $C_{in}$ is the number of channels the image has, and $H_{in}, W_{in}$ are the image height and width respectively.\n", "\n", "We can modify the convolution to have different properties with the parameters:\n", "- kernel_size\n", "- stride\n", "- padding\n", "\n", "They can change the output dimension so be careful.\n", "\n", "See the [`torch.nn.Conv2d` docs](https://pytorch.org/docs/stable/nn.html#torch.nn.Conv2d) for more information." ] }, { "cell_type": "markdown", "id": "pacific-stamp", "metadata": { "id": "a2rwq0B44sqg" }, "source": [ "To illustrate what the `Conv2d` module is doing, let's set the conv weights manually to a Gaussian blur kernel.\n", "\n", "We can see that it applies the kernel to the image." ] }, { "cell_type": "code", "execution_count": 20, "id": "lonely-roommate", "metadata": { "id": "L_vmY1YF4sqg" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAEICAYAAACZA4KlAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAARmElEQVR4nO3dfbBcdX3H8feHkAQhIQlCQhqCF3mwgGODXsGCraEoQnwAa6XEoimDhiLaUhh8wLEiYx1gEEbLg4ZCARUQRx6CDRUMUqqDwI0iCQZ5MpKYay4kMASU5Cb59o89OJtwz9mb3bN79t7f5zVz5+493/Pw3U0+e87ueVJEYGaj3w5VN2BmneGwmyXCYTdLhMNulgiH3SwRDrtZIhz2EUDSNyR9oexxG8ynR1JI2jGn/oik2a0uxzpH3s9uQ5HUA/wGGBsRmypux0rgNXuXkzSm6h5sdHDYKyDpQEn3SHo+2xx+f13tGklXSFok6SXgyGzYl+vG+bSkfkmrJX0s29zer276L2ePZ0taJeksSQPZNCfXzec9kn4h6QVJKyWdux3PYYWkd2aPz5X0PUnflrRe0lJJB0j6XLbclZKOrpv2ZEnLs3GfknTqNvMuen7jJV0k6WlJa7KPLa/Z3n+DFDnsHSZpLHA7cCcwFfgU8B1Jb6gb7cPAvwMTgZ9sM/0xwJnAO4H9gHc0WOSewCRgBnAKcJmkKVntJeCjwGTgPcBpko5v8qm9D/gWMAX4BfBDav+/ZgDnAd+sG3cAeC+wK3AycImkNw/z+V0AHADMyuozgH9rsuekOOyd9zZgAnB+RGyMiLuBHwBz68a5LSJ+GhFbIuLlbaY/AfiviHgkIv4AfKnB8gaB8yJiMCIWAS8CbwCIiHsiYmm2nIeBG2j85pHn/yLih9nn++8Be2TPcRC4EeiRNDlb7n9HxJNR87/U3vj+qtHzkyTg48C/RsS6iFgPfAU4scmekzLkN63WVn8GrIyILXXDfkttDfWKlQ2m7xvmuABrt/mC7Q/U3myQdBhwPvBGYBwwnlpQm7Gm7vEfgWcjYnPd32TLfV7SscAXqa2hdwB2BpZm4xQ9vz2ycZfUcg+AAH+vMQxes3feamCmpPrXfm/gd3V/F+0i6Qf2qvt7Zgu9XA8sBGZGxCTgG9TC0zaSxgPfBy4CpkXEZGBR3XKLnt+z1N44Do6IydnPpIiY0M6eRwuHvfPup/ZZ+dOSxmb7qt9HbVN3OG4CTs6+5NuZ1j6vTgTWRcTLkg6l9l1Bu72yBfEMsClbyx9dV899ftnW0JXUPuNPBZA0Q9K7O9D3iOewd1hEbATeDxxLbU11OfDRiHh0mNPfAXwd+DHwBHBfVtrQRDufAM6TtJ5aqG5qYh7bJfuc/c/Zsp6j9gazsK7e6Pl9Jhv+M0kvAD8i+w7CivmgmhFO0oHAMmD8aDz4ZbQ/v07ymn0EkvQBSeOyXWgXALePpiCM9udXFYd9ZDqV2mfeJ4HNwGnVtlO60f78KuHNeLNEeM1uloiOHlQzTuNjJ3bp5CLNkvIyL7ExNgx5rERLYc+OY/4atSOY/jMizi8afyd24TAd1coizazA/bE4t9b0Znx26uVl1PYXHwTMlXRQs/Mzs/Zq5TP7ocATEfFUdqDIjcBx5bRlZmVrJewz2PokhVVsfTIHAJLmS+qT1DfY1EFeZlaGVsI+1JcAr9qPFxELIqI3InrHMr6FxZlZK1oJ+yq2PiNpL2pndJlZF2ol7A8C+0vaR9I4ahcQWNhgGjOrSNO73iJik6RPUrv80Bjg6oh4pLTOzKxULe1nzy5ztKikXsysjXy4rFkiHHazRDjsZolw2M0S4bCbJcJhN0uEw26WCIfdLBEOu1kiHHazRDjsZolw2M0S4bCbJcJhN0uEw26WCIfdLBEOu1kiHHazRDjsZolw2M0S4bCbJcJhN0uEw26WCIfdLBEOu1kiHHazRDjsZolw2M0S4bCbJcJhN0tES7dslrQCWA9sBjZFRG8ZTZlZ+VoKe+bIiHi2hPmYWRt5M94sEa2GPYA7JS2RNH+oESTNl9QnqW+QDS0uzsya1epm/BERsVrSVOAuSY9GxL31I0TEAmABwK7aLVpcnpk1qaU1e0Sszn4PALcAh5bRlJmVr+mwS9pF0sRXHgNHA8vKaszMytXKZvw04BZJr8zn+oj4n1K6MrPSNR32iHgK+IsSezGzNvKuN7NEOOxmiXDYzRLhsJslwmE3S0QZJ8JYxfrPPDy3pgbHLO60tniE5/68ePrp920unv/tDxTPwDrGa3azRDjsZolw2M0S4bCbJcJhN0uEw26WCIfdLBGjZj/7wOn5+5oBnn/TYGH9lqMvLbOdjjpw3INNT/tybCqsT9rhNYX1gY+8VFhf/fX8/2IX//5dhdOuPWHXwvqmlasK67Y1r9nNEuGwmyXCYTdLhMNulgiH3SwRDrtZIhx2s0QoonM3adlVu8VhOqrp6R+78q25tUfnXF447XiNbXq5Vo2TVswurD/34Qb74Vc8XWI3I8P9sZgXYp2GqnnNbpYIh90sEQ67WSIcdrNEOOxmiXDYzRLhsJslYkSdz37Fkdfl1hrtR79g7f6F9YGNE5vqqQw3L3lLYX3v24fcbdoVVh1VvL64cM71ubUPTnihcNpv99xTWD/p+tmF9ef+fq/cWornwjdcs0u6WtKApGV1w3aTdJekx7PfU9rbppm1ajib8dcAx2wz7LPA4ojYH1ic/W1mXaxh2CPiXmDdNoOPA67NHl8LHF9uW2ZWtma/oJsWEf0A2e+peSNKmi+pT1LfIBuaXJyZtart38ZHxIKI6I2I3rGMb/fizCxHs2FfI2k6QPZ7oLyWzKwdmg37QmBe9ngecFs57ZhZuzQ8n13SDcBsYHdgDfBF4FbgJmBv4GngQxGx7Zd4r9Lq+ex6y8G5tWdnFZ/bPPXWXxfWN69t2L41YYc35d/g/b03/rRw2tMnr2xp2W+46rTcWs8X7mtp3t2q6Hz2hgfVRMTcnFLzqTWzjvPhsmaJcNjNEuGwmyXCYTdLhMNulogRdSlpG13WfvwvC+t9X7qipfkv2bAxt3bOPoe2NO9u5UtJm5nDbpYKh90sEQ67WSIcdrNEOOxmiXDYzRLhsJslwmE3S4TDbpYIh90sEQ67WSIcdrNEOOxmiXDYzRIxom7ZbCPPqnMOz61tOWR9W5c9bUz++eyb/qb4Ntk73r2k7HYq5zW7WSIcdrNEOOxmiXDYzRLhsJslwmE3S4TDbpYIXzd+FNjx9T25tSdOmV447eUnLii5m63N3mkwtzZG1a1rnhx8sbD+ide9vUOdlKul68ZLulrSgKRldcPOlfQ7SQ9lP3PKbNjMyject9ZrgGOGGH5JRMzKfhaV25aZla1h2CPiXmBdB3oxszZq5UPTJyU9nG3mT8kbSdJ8SX2S+gbZ0MLizKwVzYb9CmBfYBbQD3w1b8SIWBARvRHRO5bxTS7OzFrVVNgjYk1EbI6ILcCVwOi8JabZKNJU2CXV78/5ALAsb1wz6w4Nz2eXdAMwG9hd0irgi8BsSbOAAFYAp7avxdHvxQ8dVlh/5s3F78nn/e2NubUTJz7XVE/l6c7jtt75ozMK6wfQ15lGOqhh2CNi7hCDr2pDL2bWRt35tmtmpXPYzRLhsJslwmE3S4TDbpYIX0q6BDrk4ML65Ev7C+uLeq4orLfzVNBbX5pQWF/2x71amv8PLpydWxuzofj06nnn3V5Ynz9pdTMtATDu92Obnnak8prdLBEOu1kiHHazRDjsZolw2M0S4bCbJcJhN0uE97MP02+/lH/r4S+c+N3Caf9h4trC+tOb/lBYf3Rj7lW/APjUDR/Lre3cP+RVhf9k+j3PFtY3/+qxwnojk/hZ09M+/rlpDWZevJ/9NwWXi+65rfhS0qOR1+xmiXDYzRLhsJslwmE3S4TDbpYIh90sEQ67WSK8n32YJr91ILfWaD/6Ub96f2F98D/2LKy/5rYHCus93FdYL7K56Slbt+UdhxTWj5/c6CLGxeuqdVvG5RcfWNpg3qOP1+xmiXDYzRLhsJslwmE3S4TDbpYIh90sEQ67WSKGc8vmmcB1wJ7AFmBBRHxN0m7Ad4EeardtPiEiqr4/cNu89pT885/3O/O0wmn3Pbt4P/iOPN1UTyPdcwfsVFg/YqfW1kXzl52UW9ud1s7TH4mG82puAs6KiAOBtwGnSzoI+CywOCL2BxZnf5tZl2oY9ojoj4ifZ4/XA8uBGcBxwLXZaNcCx7epRzMrwXZtJ0nqAQ4B7gemRUQ/1N4QgKmld2dmpRl22CVNAL4PnBERL2zHdPMl9UnqG2RDMz2aWQmGFXZJY6kF/TsRcXM2eI2k6Vl9OjDkmSIRsSAieiOidyzjy+jZzJrQMOySBFwFLI+Ii+tKC4F52eN5wG3lt2dmZRnOKa5HAB8Blkp6KBt2DnA+cJOkU4CngQ+1pcMusan/97m1fc/Or1m+tW/d1NL0yzcWX4J74uWTWpr/aNMw7BHxEyDv4uNHlduOmbWLj6AzS4TDbpYIh90sEQ67WSIcdrNEOOxmifClpK2t3r0s/8jqWyZf1mDqgktBA/MemVdYn3LHgw3mnxav2c0S4bCbJcJhN0uEw26WCIfdLBEOu1kiHHazRHg/u7XV3+36cG5t5x0mFE772OBLhfWdL53cTEvJ8prdLBEOu1kiHHazRDjsZolw2M0S4bCbJcJhN0uE97NbSwY+cXhhfdqY/HPKfzOYfxtsgLlfObuwvvsdxbfCtq15zW6WCIfdLBEOu1kiHHazRDjsZolw2M0S4bCbJaLhfnZJM4HrgD2BLcCCiPiapHOBjwPPZKOeExGL2tWoVUPjxxfWP/hPdxfW12/ZmFub88BphdPu/U3vRy/TcA6q2QScFRE/lzQRWCLprqx2SURc1L72zKwsDcMeEf1Af/Z4vaTlwIx2N2Zm5dquz+ySeoBDgPuzQZ+U9LCkqyVNyZlmvqQ+SX2DbGitWzNr2rDDLmkC8H3gjIh4AbgC2BeYRW3N/9WhpouIBRHRGxG9Yyn+/Gdm7TOssEsaSy3o34mImwEiYk1EbI6ILcCVwKHta9PMWtUw7JIEXAUsj4iL64ZPrxvtA8Cy8tszs7IM59v4I4CPAEslPZQNOweYK2kWEMAK4NQ29GdV2xKF5W/dfmRh/Y5fzs6t7X3Tz5poyJo1nG/jfwJoiJL3qZuNID6CziwRDrtZIhx2s0Q47GaJcNjNEuGwmyXCl5K2QjGYf4oqQM/nfRrqSOE1u1kiHHazRDjsZolw2M0S4bCbJcJhN0uEw26WCEUUn69c6sKkZ4Df1g3aHXi2Yw1sn27trVv7AvfWrDJ7e11E7DFUoaNhf9XCpb6I6K2sgQLd2lu39gXurVmd6s2b8WaJcNjNElF12BdUvPwi3dpbt/YF7q1ZHemt0s/sZtY5Va/ZzaxDHHazRFQSdknHSPq1pCckfbaKHvJIWiFpqaSHJPVV3MvVkgYkLasbtpukuyQ9nv0e8h57FfV2rqTfZa/dQ5LmVNTbTEk/lrRc0iOS/iUbXulrV9BXR163jn9mlzQGeAx4F7AKeBCYGxG/6mgjOSStAHojovIDMCT9NfAicF1EvDEbdiGwLiLOz94op0TEZ7qkt3OBF6u+jXd2t6Lp9bcZB44H/pEKX7uCvk6gA69bFWv2Q4EnIuKpiNgI3AgcV0EfXS8i7gXWbTP4OODa7PG11P6zdFxOb10hIvoj4ufZ4/XAK7cZr/S1K+irI6oI+wxgZd3fq+iu+70HcKekJZLmV93MEKZFRD/U/vMAUyvuZ1sNb+PdSdvcZrxrXrtmbn/eqirCPtStpLpp/98REfFm4Fjg9Gxz1YZnWLfx7pQhbjPeFZq9/Xmrqgj7KmBm3d97Aasr6GNIEbE6+z0A3EL33Yp6zSt30M1+D1Tcz5900228h7rNOF3w2lV5+/Mqwv4gsL+kfSSNA04EFlbQx6tI2iX74gRJuwBH0323ol4IzMsezwNuq7CXrXTLbbzzbjNOxa9d5bc/j4iO/wBzqH0j/yTw+Sp6yOnr9cAvs59Hqu4NuIHaZt0gtS2iU4DXAouBx7Pfu3VRb98ClgIPUwvW9Ip6ezu1j4YPAw9lP3Oqfu0K+urI6+bDZc0S4SPozBLhsJslwmE3S4TDbpYIh90sEQ67WSIcdrNE/D9uT43qYXyQkgAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP0AAAEICAYAAACUHfLiAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAUQ0lEQVR4nO3dfZBddX3H8fdnN7sJySYkISEkG8pDeIapASOKOJUWREQt0Kmt2FF01FBHR2ydKUgdZTp0io4PtTMd2yAIImqpqNAZWo2RltqpaKAxDwQMhIRsEvJAEvJE9vHbP/bszBX2/s5m7929d/P7vGZ29t7zPfs7372bT86995z7O4oIzCwfLY1uwMzGl0NvlhmH3iwzDr1ZZhx6s8w49GaZcegbRNImSVdUqV0mqWu8e6rY/qmSQtKkKvV1ki4b366sXob9o5qlRMT5je7BRs97+mPMcHvnantsy5ND31hvkPSUpL2SvilpynArFU+1z6i4f4+k24vbl0nqknSzpBeBb0q6TdL3JX1b0n7gg5KOl3SXpO2Stkq6XVJrMUarpC9J2i1pI/DOVNOVL02Kbf1rsa0DktZIOkvSZyTtlLRF0pUVP/shSeuLdTdKuvFVY/9V0eM2SR+p/N0lTS76fEHSDkn/JOm4UT3yGXPoG+vPgLcDi4CzgM+OcpyTgNnAKcDSYtk1wPeBmcD9wL1AH3AGcCFwJfCRYt2PAu8qli8B/vgot/9u4D5gFvB/wI8Z/LfVCfwN8M8V6+4stjUD+BDwVUkXAUi6CvhL4Iqiz7e+ajtfYPBxWlzUO4HPHWWvFhH+asAXsAn484r7VwPPFbcvA7oqagGcUXH/HuD2inV7gCkV9duAxyruzwO6geMqll0PPFrc/tmrermy2OakRO9XVGxreUXt3cBBoLW4P70Ya2aVsX4E3FTcvhv4u4raGUO/OyDgELCoon4J8Hyj/5YT7cuv9RprS8XtzcCCUY6zKyKOJMY+BWgDtksaWtZSsc6CYXo5Gjsqbr8C7I6I/or7AB3APknvAD7P4B67BZgKrKnoY2WV32Fuse4TFb+DgNaj7DV7Dn1jnVxx+3eAbVXWO8zgP/ghJwGVh/SG+6hk5bItDO7p50RE3zDrbh+ml7qTNBl4EPgA8FBE9Er6EYPhHepjYcWPVPa0m8H/QM6PiK1j0V8u/Jq+sT4uaaGk2cCtwL9UWW8V8L7iDbereO1r3aSI2A78BPiypBmSWiQtkjQ0zgPAJ4teZgG3jOq3KdcOTAZ2AX3FXv/KivoDwIcknStpKhWv1yNiALiTwfcATgSQ1Cnp7WPU6zHLoW+s7zAYxo3F1+1V1ruJwdfK+xh88+9Ho9jWBxgM3VPAXgbf5Jtf1O5k8M23XwNPAj8YxfilIuIA8EkGw70XeB/wcEX934F/AB4FngX+tyh1F99vLpb/ojgq8VPg7LHo9Vim4g0Rs6Yj6VxgLTC5yssSGwXv6a2pSLpOUnvxMuMLwL858PXl0FuzuZHB1/zPAf3AxxrbzrHHT+/NMuM9vVlmxvU4fbsmxxSmjecmzbJyhEP0RLdS69QU+uKY8dcYPCvqGxFxR2r9KUzjjbq8lk2aWcLjsaJ0nVE/vS8+ofWPwDuA84DrJZ032vHMbHzU8pr+YuDZiNgYET3A9xj8ZJeZNbFaQt/Jb38goqtYZmZNrJbX9MO9WfCa43+SllJ8xnvKb31mxMwaoZY9fRe//SmohQzzKbGIWBYRSyJiSRuTa9icmdVDLaH/FXCmpNMktQPvpeLDE2bWnEb99D4i+iR9gsFPZ7UCd0fEurp1ZmZjoqbj9BHxCPBInXoxs3Hg03DNMuPQm2XGoTfLjENvlhmH3iwzDr1ZZhx6s8w49GaZcejNMuPQm2XGoTfLjENvlhmH3iwzDr1ZZhx6s8w49GaZcejNMuPQm2XGoTfLjENvlhmH3iwzDr1ZZhx6s8w49GaZcejNMuPQm2XGoTfLjENvlhmH3iwzDr1ZZhx6s8w49GaZmdToBmx4LVOmlK8za2Z6hcnt9WkmIQ69UrrOwN696TH6+urVjo1ATaGXtAk4APQDfRGxpB5NmdnYqcee/vcjYncdxjGzceDX9GaZqTX0AfxE0hOSlg63gqSlklZKWtlLd42bM7Na1fr0/tKI2CbpRGC5pKcj4rHKFSJiGbAMYIZmR43bM7Ma1bSnj4htxfedwA+Bi+vRlJmNnVGHXtI0SdOHbgNXAmvr1ZiZjY1ant7PA34oaWic70TEf9TSjCZPLl2nde6cZL1/7szSMXpOSB8DH5jc+Pc3u2e0lq5zsDPdZ/9xtfcRStcnpw/BAzC9qz9Zn9Z1uHSMlue3Jev9L+1JDxB+ZTlk1KGPiI3A6+rYi5mNg8bv0sxsXDn0Zplx6M0y49CbZcahN8uMQ2+WGYfeLDPjOomGWlpomTqtar3/dWeUjrH9jdV/HmD/Ob2lY8w7OX1GyZyph0rHGGu/M7m8h7Om7UzWj28tn+CiTIsGkvWt3bNKx1j9cmeyvu43C0vHmL/izGR91n89n6z37Ug/VkA2J/B4T2+WGYfeLDMOvVlmHHqzzDj0Zplx6M0y49CbZWZ8L3bR3oZOqX7MdvM7p5YO8ebL1yTrV8x6qnSMU9t2JevTW3pKxyhzJNKTYGzsOTFZ39yTniwE4MhAW7K+py99TsNITG89kqxf3LGxdIz3zvplsv70gnmlY3xu5h8m69FyWrI++9HSTZQfyz9GjuN7T2+WGYfeLDMOvVlmHHqzzDj0Zplx6M0y49CbZWZcj9NHixiY2l613jsj/dltgO7+dMv3bX1T6Rg7D3Yk6/0Dtf9f2N2T7rNnR/qchOO2lV/sYlL5NSJqVnbBjMMnpy9kAXDR7z6XrH9sQflB9L9d/FCyfnPfHyXr7QdPKd3G1P9MP6ADBw6UjjEReE9vlhmH3iwzDr1ZZhx6s8w49GaZcejNMuPQm2XGoTfLzPhe7KK3n9bte6rWO39WPunDM2vPSdan7Ck/wWfWnr5kXQO1T5ag3nQfk/a9nKy37Cs/ESS6a5/so4za0xN19J9UfrGLZ998VrL+xWunlI7xhdMfTNb/YvGKZP3vX3hX6TbOfPak9ArrMzk5R9LdknZKWluxbLak5ZI2FN/L//Jm1hRG8vT+HuCqVy27BVgREWcCK4r7ZjYBlIY+Ih4DXv2c/Brg3uL2vcC19W3LzMbKaN/ImxcR2wGK71VneZS0VNJKSSt7Bmq/oKKZ1WbM372PiGURsSQilrS3lHxky8zG3GhDv0PSfIDi+wiuA2xmzWC0oX8YuKG4fQOQ/rCzmTWN0uP0kr4LXAbMkdQFfB64A3hA0oeBF4D3jGRj0dtL3/YdVesdPy0/Djq95LhxHCqfWWLgSPoCDuOh7GyC8rMNmkTi7zlkfveZyfqGUxaWjvH0wvQx9Gs61ifrd59/Sek2uufPSNYnpTcxYZSGPiKur1K6vM69mNk48Gm4Zplx6M0y49CbZcahN8uMQ2+WGYfeLDMOvVlmxnUSDQAGql8R5Vi5gkhOWkpOlgKIKel/ZupX6RhHIr2dua2Tk/XTZlafvGXIi7PS00K0TUr/HtGXnpylWXhPb5YZh94sMw69WWYcerPMOPRmmXHozTLj0JtlZvyP09vIqPzYdev06ekVOueVjtFzUnqMvimtyXr3zHQdYO+56d+l88LtpWOc055eZxLpPi46fkvpNu4/K31RjuPnpyfy6NvSVbqNZuA9vVlmHHqzzDj0Zplx6M0y49CbZcahN8uMQ2+WGR+nHyNqa0/WW2bPTNZj/pzSbew9N31xht0Xlh/r7zh7b7J+YsfBZH3BlEOl2/jTmc8n62+euqF0jPPaqs/DANCqKcn6H3Q8VbqNb5x7abLe1zk7PYCP05tZM3LozTLj0JtlxqE3y4xDb5YZh94sMw69WWYcerPM+OSc4bSkJ2RonXtC6RA95y1M1ndfkD6ZZP9Z6ZNRAE47d1uyfkvnL0vHeP2Uzcn6ZKX72DUwtXQbG7rTk0883T2/dIyZLek+F2mgdIwyUXLRDfVH+udr7mB8lO7pJd0taaektRXLbpO0VdKq4uvqsW3TzOplJE/v7wGuGmb5VyNicfH1SH3bMrOxUhr6iHgMKL8QmJlNCLW8kfcJSauLp/9Vr/wnaamklZJW9tJdw+bMrB5GG/qvA4uAxcB24MvVVoyIZRGxJCKWtJG+sqiZjb1RhT4idkREf0QMAHcCF9e3LTMbK6MKvaTKYyzXAWurrWtmzaX0OL2k7wKXAXMkdQGfBy6TtJjBQ5ObgBvHrsX6a5maPras005O1ndeUjKZArD3rUeS9bef82SyftbUF0u30VZyDL2rp7zP5XvOS9a3HJiZrO946fjSbbRuSp+T0NtRfoT7urekzzn4zIn/nayv615Uuo32rSUTn+xNv59dfmZFcygNfURcP8ziu8agFzMbBz4N1ywzDr1ZZhx6s8w49GaZcejNMuPQm2XGoTfLTJaTaLTMm5usd70tPUlGx1XlJ858YP7Tyfr+vvQJK9/bvKR0Gzs3pvuc2pWeDARg2tb0iTHH7e5L1k/f11O6jUl7diXrey8qv5rP6gs6k/XDc9O/x5rD6UlNADq2lKzw0r7SMSYC7+nNMuPQm2XGoTfLjENvlhmH3iwzDr1ZZhx6s8xkeZw+2tuS9Z6Z6Z9vaym/sMK316VnEDvuifREHrOf7i3dxtlbXk7WW/YcKB1j4OX96frBg+kBYgSXeJiZnmijb0r6vAmA+VPTv2tvSRtr9i4o3UbH1vQ0GP0lj9VE4T29WWYcerPMOPRmmXHozTLj0JtlxqE3y4xDb5aZLI/Ts/OlZHne41WvxwnA/hfnJ+sAnZvTn0Oftmpjst6/a3fpNgb60tsoP5tgfOj4Gcn6oQUqHWPx9K5k/bne9N9s4/PzSrdx9ouHk/UYmCiXs0jznt4sMw69WWYcerPMOPRmmXHozTLj0JtlxqE3y4xDb5aZLE/O6d+3L1mf+j/PpOtPpi9UARAH0pNP9B1OnwgyUbSeMLt0nf2vT09g0XfBodIxTmlPn6z04J70xUGOX52eOAWgteuFZD19KtTEUbqnl3SypEclrZe0TtJNxfLZkpZL2lB8T58SZWZNYSRP7/uAT0fEucCbgI9LOg+4BVgREWcCK4r7ZtbkSkMfEdsj4sni9gFgPdAJXAPcW6x2L3DtGPVoZnV0VG/kSToVuBB4HJgXEdth8D8G4MQqP7NU0kpJK3vprrFdM6vViEMvqQN4EPhURIx4WtCIWBYRSyJiSRuTR9OjmdXRiEIvqY3BwN8fET8oFu+QNL+ozwd2jk2LZlZPI3n3XsBdwPqI+EpF6WHghuL2DcBD9W/PzOptJMfpLwXeD6yRtKpYditwB/CApA8DLwDvGZMOx0LJBRr695e8eimrZ6R/UWfpOl1Xpafz+OziH5dvh/REG8ufOi9ZX7TqlfJt7E5PrnKsKA19RPwcqj7il9e3HTMbaz4N1ywzDr1ZZhx6s8w49GaZcejNMuPQm2Umy8/T28hpcvrU6f2nTSsd443n/yZZf8Nxm0rH+Nzma5L1mb9oT9bbN6QvLgLQV3LxkGOF9/RmmXHozTLj0JtlxqE3y4xDb5YZh94sMw69WWYcerPM+OQcS2pdcFKyvvec8v3G2zp2JOv37bmkdIxnfrYoWT/153uS9VwmyBgJ7+nNMuPQm2XGoTfLjENvlhmH3iwzDr1ZZhx6s8z4OL0l9c+Zkax3z05fyALgsZ1nJOtbVs8vHeP0nx5O1mPD8+l6JhNkjIT39GaZcejNMuPQm2XGoTfLjENvlhmH3iwzDr1ZZhx6s8yUnpwj6WTgW8BJwACwLCK+Juk24KPArmLVWyPikbFq1BqjZf8ryfoJqzpKx9j/1IJk/fQ16RNvAFrXpK9QM9DdXTqGDRrJGXl9wKcj4klJ04EnJC0val+NiC+NXXtmVm+loY+I7cD24vYBSeuBzrFuzMzGxlG9ppd0KnAh8Hix6BOSVku6W9KsejdnZvU34tBL6gAeBD4VEfuBrwOLgMUMPhP4cpWfWypppaSVvfh1l1mjjSj0ktoYDPz9EfEDgIjYERH9ETEA3AlcPNzPRsSyiFgSEUvaSF/22MzGXmnoJQm4C1gfEV+pWF75ecjrgLX1b8/M6m0k795fCrwfWCNpVbHsVuB6SYuBADYBN45Bf2ZWZ4qI8duYtAvYXLFoDrB73BoYPfdZXxOhz4nQI7y2z1MiYm7qB8Y19K/ZuLQyIpY0rIERcp/1NRH6nAg9wuj69Gm4Zplx6M0y0+jQL2vw9kfKfdbXROhzIvQIo+izoa/pzWz8NXpPb2bjzKE3y0zDQi/pKknPSHpW0i2N6qOMpE2S1khaJWllo/sZUnzIaaektRXLZktaLmlD8b2hH4Kq0uNtkrYWj+cqSVc3sseip5MlPSppvaR1km4qljfb41mtz6N6TBvyml5SK/Ab4G1AF/Ar4PqIeGrcmykhaROwJCKa6kQNSb8HHAS+FREXFMu+COyJiDuK/0hnRcTNTdbjbcDBZpqHoTilfH7lnBHAtcAHaa7Hs1qff8JRPKaN2tNfDDwbERsjogf4HnBNg3qZkCLiMWDPqxZfA9xb3L6XwX8QDVOlx6YTEdsj4sni9gFgaM6IZns8q/V5VBoV+k5gS8X9Lpp3Yo4AfiLpCUlLG91MiXnFpCdDk5+c2OB+qmnaeRheNWdE0z6etcxt0ajQa5hlzXrs8NKIuAh4B/Dx4imrjd6I5mFohGHmjGhKo53bYkijQt8FnFxxfyGwrUG9JEXEtuL7TuCHVJk3oEnsGPrIc/F9Z4P7eY2RzsMw3oabM4ImfDxrmdtiSKNC/yvgTEmnSWoH3gs83KBeqpI0rXjDBEnTgCtp7nkDHgZuKG7fADzUwF6G1YzzMFSbM4ImezzrNrdFRDTkC7iawXfwnwP+ulF9lPR4OvDr4mtdM/UJfJfBp3K9DD5z+jBwArAC2FB8n92EPd4HrAFWMxiq+U3wWL6FwZeXq4FVxdfVTfh4VuvzqB5Tn4ZrlhmfkWeWGYfeLDMOvVlmHHqzzDj0Zplx6M0y49CbZeb/AYbPJTlWdWLCAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# an entire mnist digit\n", "image = np.array([0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0.3803922 , 0.37647063, 0.3019608 ,0.46274513, 0.2392157 , 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0.3529412 , 0.5411765 , 0.9215687 ,0.9215687 , 0.9215687 , 0.9215687 , 0.9215687 , 0.9215687 ,0.9843138 , 0.9843138 , 0.9725491 , 0.9960785 , 0.9607844 ,0.9215687 , 0.74509805, 0.08235294, 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.54901963,0.9843138 , 0.9960785 , 0.9960785 , 0.9960785 , 0.9960785 ,0.9960785 , 0.9960785 , 0.9960785 , 0.9960785 , 0.9960785 ,0.9960785 , 0.9960785 , 0.9960785 , 0.9960785 , 0.9960785 ,0.7411765 , 0.09019608, 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0.8862746 , 0.9960785 , 0.81568635,0.7803922 , 0.7803922 , 0.7803922 , 0.7803922 , 0.54509807,0.2392157 , 0.2392157 , 0.2392157 , 0.2392157 , 0.2392157 ,0.5019608 , 0.8705883 , 0.9960785 , 0.9960785 , 0.7411765 ,0.08235294, 0., 0., 0., 0.,0., 0., 0., 0., 0.,0.14901961, 0.32156864, 0.0509804 , 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.13333334,0.8352942 , 0.9960785 , 0.9960785 , 0.45098042, 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0.32941177, 0.9960785 ,0.9960785 , 0.9176471 , 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0.32941177, 0.9960785 , 0.9960785 , 0.9176471 ,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0.4156863 , 0.6156863 ,0.9960785 , 0.9960785 , 0.95294124, 0.20000002, 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0.09803922, 0.45882356, 0.8941177 , 0.8941177 ,0.8941177 , 0.9921569 , 0.9960785 , 0.9960785 , 0.9960785 ,0.9960785 , 0.94117653, 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0.26666668, 0.4666667 , 0.86274517,0.9960785 , 0.9960785 , 0.9960785 , 0.9960785 , 0.9960785 ,0.9960785 , 0.9960785 , 0.9960785 , 0.9960785 , 0.5568628 ,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0.14509805, 0.73333335,0.9921569 , 0.9960785 , 0.9960785 , 0.9960785 , 0.8745099 ,0.8078432 , 0.8078432 , 0.29411766, 0.26666668, 0.8431373 ,0.9960785 , 0.9960785 , 0.45882356, 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0.4431373 , 0.8588236 , 0.9960785 , 0.9490197 , 0.89019614,0.45098042, 0.34901962, 0.12156864, 0., 0.,0., 0., 0.7843138 , 0.9960785 , 0.9450981 ,0.16078432, 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0.6627451 , 0.9960785 ,0.6901961 , 0.24313727, 0., 0., 0.,0., 0., 0., 0., 0.18823531,0.9058824 , 0.9960785 , 0.9176471 , 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0.07058824, 0.48627454, 0., 0.,0., 0., 0., 0., 0.,0., 0., 0.32941177, 0.9960785 , 0.9960785 ,0.6509804 , 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0.54509807, 0.9960785 , 0.9333334 , 0.22352943, 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0.8235295 , 0.9803922 , 0.9960785 ,0.65882355, 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0.9490197 , 0.9960785 , 0.93725497, 0.22352943, 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0.34901962, 0.9843138 , 0.9450981 ,0.3372549 , 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.01960784,0.8078432 , 0.96470594, 0.6156863 , 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0.01568628, 0.45882356, 0.27058825,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0., 0.,0., 0., 0., 0.], dtype=np.float32)\n", "image_torch = torch.from_numpy(image).view(1, 1, 28, 28)\n", "\n", "# a gaussian blur kernel\n", "gaussian_kernel = torch.tensor([[1., 2, 1],[2, 4, 2],[1, 2, 1]]) / 16.0\n", "\n", "conv = nn.Conv2d(1, 1, 3)\n", "# manually set the conv weight\n", "conv.weight.data[:] = gaussian_kernel\n", "\n", "convolved = conv(image_torch)\n", "\n", "plt.title('original image')\n", "plt.imshow(image_torch.view(28,28).detach().numpy())\n", "plt.show()\n", "\n", "plt.title('blurred image')\n", "plt.imshow(convolved.view(26,26).detach().numpy())\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "ongoing-hometown", "metadata": { "id": "xmtCp7r84sqh" }, "source": [ "As we can see, the image is blurred as expected. \n", "\n", "In practice, we learn many kernels at a time. In this example, we take in an RGB image (3 channels) and output a 16 channel image. After an activation function, that could be used as input to another `Conv2d` module." ] }, { "cell_type": "code", "execution_count": 21, "id": "primary-distributor", "metadata": { "id": "QjcwK2Pa4sqh" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "im shape torch.Size([4, 3, 32, 32])\n", "convolved im shape torch.Size([4, 16, 30, 30])\n" ] } ], "source": [ "im_channels = 3 # if we are working with RGB images, there are 3 input channels, with black and white, 1\n", "out_channels = 16 # this is a hyperparameter we can tune\n", "kernel_size = 3 # this is another hyperparameter we can tune\n", "batch_size = 4\n", "image_width = 32\n", "image_height = 32\n", "\n", "im = torch.randn(batch_size, im_channels, image_width, image_height)\n", "\n", "m = nn.Conv2d(im_channels, out_channels, kernel_size)\n", "convolved = m(im) # it is a module so we can call it\n", "\n", "print('im shape', im.shape)\n", "print('convolved im shape', convolved.shape)" ] }, { "cell_type": "markdown", "id": "motivated-leadership", "metadata": {}, "source": [ "## Recurrent Neural Networks\n", "When working with text, we often want to use text embeedings and recurrent neural networks. This is because we have to:\n", "\n", "1. Represent words (or characters) as a vector.\n", "2. Typically meaning of a word depends on a context (\"March is a cold month.\" vs \"Their march was very loud.\")\n", "\n", "PyTorch implements these concept using [`torch.nn.Embedding`](https://pytorch.org/docs/stable/generated/torch.nn.Embedding.html#torch.nn.Embedding) and [`torch.nn.RNN`](https://pytorch.org/docs/stable/generated/torch.nn.RNN.html#torch.nn.RNN).\n", "\n", "Embedding converts a list of $n$ words into an $\\mathbb{R}^{n \\times d}$ matrix, where each word is respresented by $\\mathbb{R}^d$ vector. When creating it you need to provide number of words in your dictionary, and the embedding size: $d$.\n", "\n", "RNN allows us to use *context* by having the output $y$ be the function of input $x$, which we provide and the hidden state, which depends on previous inputs in a sequence. The layer takes input of (sequence length, batch size, $d_{in}$), and an optional argument for hidden state. It's output is a tuple containing actual output (sequence length, batch size, $d_{out}$), and hidden state.\n", "\n", "For a longer explanation [take a look a this post](http://colah.github.io/posts/2015-08-Understanding-LSTMs/). While it's on topic of LSTMs the beginning does a good explanation of how RNNs work.\n", "If you really want to take a deep dive [this post](http://karpathy.github.io/2015/05/21/rnn-effectiveness/) goes in-depth on various applications of RNNs and LSTMs.\n", "\n", "Lastly if that's something that you are interested in consider CSE 447 (NLP)." ] }, { "cell_type": "code", "execution_count": 22, "id": "invisible-database", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Input Data shape: torch.Size([2, 1])\n", "After Embedding shape: torch.Size([2, 1, 3])\n", "After RNN output shape: torch.Size([2, 1, 5])\n", "After RNN hidden shape: torch.Size([1, 1, 5])\n" ] } ], "source": [ "# Feel free to play with parameters\n", "embedding_size = 3\n", "num_unique_words = 10\n", "sentence_length = 2\n", "batch_size = 1\n", "hidden_size = 5\n", "\n", "# Let's generate data of shape (sentence_length, batch_size)\n", "data = torch.randint(high=num_unique_words, size=(sentence_length, batch_size))\n", "\n", "embedding = nn.Embedding(num_unique_words, embedding_size)\n", "rnn_layer = nn.RNN(embedding_size, hidden_size)\n", "\n", "print(f\"Input Data shape: {data.shape}\")\n", "embedded_vec = embedding(data)\n", "print(f\"After Embedding shape: {embedded_vec.shape}\")\n", "result, hidden = rnn_layer(embedded_vec)\n", "print(f\"After RNN output shape: {result.shape}\") # (sequence length, batch size, hidden_size)\n", "print(f\"After RNN hidden shape: {hidden.shape}\") # (# layers, batch size, hidden_size)" ] }, { "cell_type": "markdown", "id": "major-balance", "metadata": {}, "source": [ "### Beyond - More advanced example. Predicting next word." ] }, { "cell_type": "code", "execution_count": 23, "id": "sharing-istanbul", "metadata": {}, "outputs": [], "source": [ "def process_corpus(corpus, sentence_length):\n", " \"\"\"\n", " Arguments:\n", " corpus (str) -- Continous text. Can be anything but should be relatively long.\n", " sentence_length (int) -- Size of each sentence in the output.\n", " Does not have to be divisible by # of words in corpus, in which case end will be padded.\n", " Returns:\n", " Tuple of size 4 containing:\n", " - Train Input - shape (batch, sentence) containing indexes of words for each sentence.\n", " - Train Truth - Same as Train Input but contains index of the next word in a given sentence.\n", " - Word to Index Dictionary - Dictionary for each word containing a corresponding integer.\n", " - Index to Word Dictionary - Reverse of Word to Index Dictionary.\n", " \n", " Example:\n", " process_corpus(\"Sam likes cats\", 2) outputs:\n", " - [[1, 2], [3, 0]]\n", " - [[2, 3], [0, 0]]\n", " - {\"\": 0, \"Sam\": 1, \"likes\": 2, \"cats\": 3}\n", " - {0: \"\", 1: \"Sam\", 2: \"likes\", 3: \"cats\"}\n", " \"\"\"\n", " # Let's make corpus a list of words\n", " corpus = corpus.split()\n", " # QUESTION: Should we also trim/lowercase the words here? Is \"You,\" vs. \"you\" very different?\n", "\n", " # Then split it into smaller sentences of size sentence_length\n", " x = []\n", " y = []\n", " for idx in range(0, len(corpus), sentence_length):\n", " x.append(corpus[idx: idx + sentence_length])\n", " # Since we are trying to predict the next word y's are just x's shifted by one\n", " y.append(corpus[idx + 1: idx + sentence_length + 1])\n", " # Last sentences might be shorter. Let's pad it with something smaller\n", " x[-1] += [\"\" for _ in range(sentence_length - len(x[-1]))]\n", " y[-1] += [\"\" for _ in range(sentence_length - len(y[-1]))]\n", "\n", " # Create dictionary from words to indices and vice-versa\n", " # QUESTION: Is \"\" a good choice for end-of-sentence tag? Maybe we should pad beginning of the sentences too?\n", " idx_to_word = {0: \"\"}\n", " word_to_idx = {\"\": 0}\n", " idx = 1\n", " for sentence in x:\n", " for word in sentence:\n", " if word not in word_to_idx:\n", " word_to_idx[word] = idx\n", " idx_to_word[idx] = word\n", " idx += 1\n", "\n", " x_idx = torch.tensor([[word_to_idx[w] for w in s] for s in x]).long()\n", " y_idx = torch.tensor([[word_to_idx[w] for w in s] for s in y]).long()\n", "\n", " return x_idx, y_idx, word_to_idx, idx_to_word" ] }, { "cell_type": "code", "execution_count": 24, "id": "terminal-amendment", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Sequential(\n", " (0): Embedding(61, 10)\n", " (1): RNN(10, 5, batch_first=True)\n", ")\n", "0,\t4.29\n", "100,\t3.84\n", "200,\t3.41\n", "300,\t3.00\n", "400,\t2.66\n", "500,\t2.40\n", "600,\t2.18\n", "700,\t2.00\n", "800,\t1.83\n", "900,\t1.69\n" ] } ], "source": [ "# Feel free to play with parameters\n", "embedding_size = 10\n", "sentence_length = 5\n", "hidden_size = 5\n", "n_epochs = 1000\n", "\n", "# Dataset\n", "corpus = \"Hey, you. You’re finally awake. \" \\\n", " \"You were trying to cross the border, right? \" \\\n", " \"Walked right into that Imperial ambush, \" \\\n", " \"same as us, and that thief over there. \" \\\n", " \"Skyrim was fine until you came along. \" \\\n", " \"Empire was nice and lazy. \" \\\n", " \"If they hadn’t been looking for you, \" \\\n", " \"I could’ve stolen that horse and been half way to Hammerfell. \" \\\n", " \"You there. You and me — we should be here. \" \\\n", " \"It’s these Stormcloaks the Empire wants. \"\n", "\n", "x, y, _, idx_to_word = process_corpus(corpus, sentence_length)\n", "\n", "model_rnn = nn.Sequential(\n", " nn.Embedding(len(idx_to_word), embedding_size),\n", " nn.RNN(embedding_size, hidden_size, batch_first=True),\n", ")\n", "# Linear model has to be separate, because we'll be using only first output of the RNN\n", "linear = nn.Linear(hidden_size, len(idx_to_word))\n", "print(model_rnn)\n", "\n", "criterion = nn.CrossEntropyLoss()\n", "optimizer = torch.optim.Adam(list(model_rnn.parameters()) + list(linear.parameters()))\n", "\n", "for i in range(n_epochs):\n", " x_mid, _ = model_rnn(x)\n", " y_hat = linear(x_mid).transpose(1, 2) # This makes shape correct for the Loss\n", " loss = criterion(y_hat, y)\n", " optimizer.zero_grad()\n", " loss.backward()\n", " optimizer.step()\n", " \n", " if i % (n_epochs // 10) == 0:\n", " print('{},\\t{:.2f}'.format(i, loss.item()))" ] }, { "cell_type": "code", "execution_count": 25, "id": "egyptian-announcement", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Truth: ['you.', 'You’re', 'finally', 'awake.', 'You']\n", "Predict: ['', 'Empire', 'finally', 'awake.', 'You']\n" ] } ], "source": [ "# Let's see a prediction for the first sentence\n", "with torch.no_grad():\n", " y_hat = linear(model_rnn(x)[0])\n", " y_hat = torch.argmax(y_hat, dim=2)\n", " sentences_hat = [[idx_to_word[int(w)] for w in s] for s in y_hat]\n", " sentences_true = [[idx_to_word[int(w)] for w in s] for s in y]\n", "\n", " sentence_idx = 0\n", " print(f\"Truth: {sentences_true[sentence_idx]}\")\n", " print(f\"Predict: {sentences_hat[sentence_idx]}\")\n" ] }, { "cell_type": "code", "execution_count": null, "id": "initial-atlanta", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3.6.12 64-bit ('446': conda)", "language": "python", "name": "python361264bit446condade4f4cbed3884e80b0f937aab3a5c4e3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.12" } }, "nbformat": 4, "nbformat_minor": 5 }