Dynet api tutorial by woozyan · Pull Request #1312 · clab/dynet

woozyan · 2018-03-16T00:48:44Z

Rewritten the API tutorial (#1302) so that user can have a better idea on how to use DyNet masterfully.

This includes:

Create Expressions
Create Parameters/LookupParameters
Basic Expression Manipulations
Neural Networks Related Manipulations
How to use what you leaned above to create Neural Networks yourself.

woozyan · 2018-03-16T00:49:46Z

@xunzhang Please take a look of what has been modified so far. I think I will continue working on part 4 and 5 later this week.

xunzhang · 2018-03-16T00:58:29Z

Sure. This is great!

pmichel31415 · 2018-03-16T15:54:29Z

This is very cool.

I have one remark: the API for parameters was changed recently (you don't need to call dy.parameter anymore and doing so prints an error message. Should we do a mini-release including these changes and incorporate them in the tutorials/examples from now on (starting with this one). This would also simplify the examples a bit. thoughts?

xunzhang · 2018-03-16T18:50:31Z

@pmichel31415 Agree, I think this can be finished before early next week, let's make a release when we finish this. And refactor existing models with the new interface. Further implementations should base on new interfaces.

FYI, @zhechenyan please apple changes from #1233 to your API.ipynb. I'll probably fix #1306 during weekends. I'll also try to add all interface changes after v2.0.3 later. This pull request should include all those(use latest one instead of deprecated ones).

xunzhang · 2018-03-19T16:48:12Z

+    "from __future__ import division\n",
+    "from __future__ import absolute_import\n",
+    "\n",
+    "import dynet_config\n",


Add a comment that says please import dynet_config before import dynet, this is important to know.

http://dynet.readthedocs.io/en/latest/commandline.html, https://github.com/clab/dynet/blob/master/python/dynet_config.py#L1

xunzhang · 2018-03-19T16:48:28Z

+   "metadata": {},
+   "source": [
+    "#### Create Parameters\n",
+    "Parameters are things that are optimized. in contrast to a system like Torch where computational modules may have their own parameters, in DyNet parameters are just parameters."


in -> In. systems like Tensorflow and Torch.

xunzhang · 2018-03-19T16:58:04Z

-    "b = dy.parameter(pb)\n",
-    "\n",
+    "m = dy.ParameterCollection() \n",
+    "W = m.add_parameters((8,8)) # an 8x8 matrix\n",


add a comment that says add_parameters return an expr.

xunzhang · 2018-03-19T16:58:51Z

-    "p = m.add_parameters((3,5), init=dy.PyInitializer())        # Any parameter initializer\n",
+    "scale = 1\n",
+    "mean = 0\n",
+    "stddev = 1\n",


merge these three into one line please.

xunzhang · 2018-03-19T17:00:25Z

+    "p = m.add_parameters((3,5), init='he')\n",
+    "# Creates 3x5 matrix from a numpy array (size is inferred)\n",
+    "p = m.add_parameters((3,5), np.ones((3,5)))\n",
+    "# Creates 3x5 matrix from a numpy array (size is inferred)"


remove this line please.

xunzhang · 2018-03-19T17:42:50Z

+    "# initialize LookupParameters.\n",
+    "scale = 1\n",
+    "mean = 0\n",
+    "stddev = 1\n",


xunzhang · 2018-03-19T19:02:42Z

+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Expression Manipulation\n",


rename as More Expression Manipulation?

xunzhang · 2018-03-19T20:49:24Z

Just finished review above the Expression Manipulation section.

neubig · 2018-04-06T15:56:43Z

@zhechenyan @xunzhang Is this ready for a final check?

xunzhang · 2018-04-06T17:03:08Z

@neubig not yet. I'll let you know when it's ready for a final look. Probably this weekend.

xunzhang

LGTM

xunzhang · 2018-04-09T03:29:47Z

cc @neubig , please take a final review and do merge please.

neubig

Overall, looks good! I have a few comments, and also think this needs a spell check. Once that's done I think we can merge.

neubig · 2018-04-20T13:03:06Z

+   "outputs": [],
+   "source": [
+    "# create a vector/matrix expression of special values\n",
+    "# Different from TensorFlow, DyNet treats batch as the last dim\n",


This is not exactly correct. DyNet has a special "batch" dimension, so that you don't need to think about the batch dimension most of the time other than during input and output. Could we make this a little more accurate, following the explanation here?:
http://dynet.readthedocs.io/en/latest/minibatch.html

neubig · 2018-04-20T13:03:32Z

+    "dim = 5\n",
+    "batch_size = 3\n",
+    "e = dy.zeros(dim, batch_size=batch_size)\n",
+    "print('zeors of dim {} and batch_size {}:\\n{}'.format(dim, batch_size, e.npvalue()))\n",


zeors -> zeros

neubig · 2018-04-20T13:06:18Z

+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "It should be noticed that in DyNet 2.0+ and later version, the dy.parameters() is depecated so explicitly adding parameters to the computation graph is no longer necessary. Any used parameter will be added automatically."


It should be noticed that in DyNet 2.0+ and later version, the dy.parameters() is depecated
->
Previously, it was necessary to add parameters to the computation graph using "dy.parameters()", but this is now deprecated

Also it should probably be 2.0.4+ or ">2.0.3"so that people don't get confused

neubig · 2018-04-20T13:07:10Z

+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Ceate expressions from lookup parameters.\n",


Ceate -> Create

(There may be other spelling errors, so I'd try to spell-check the text)

neubig · 2018-04-20T13:08:00Z

+   "metadata": {},
+   "source": [
+    "#### More Expression Manipulation\n",
+    "DyNet provides hundreds of operations on Expression. User can manipulate Expressions, build complex Expression easily."


operations on Expression -> operations that can be applied to Expressions

User -> The user

build complex Expression -> and build complex Expressions

neubig · 2018-05-11T11:02:43Z

Hi! There are a few small comments here, but I think this is largely good. Do you think you could finish this up?

xunzhang · 2018-05-11T13:18:05Z

cc @zhechenyan

woozyan · 2018-05-11T14:03:38Z

Sure I will finish this by the end of the weekend.

neubig · 2018-06-11T18:17:37Z

Ping?

xunzhang · 2018-06-11T18:37:55Z

Ping x2

neubig · 2018-06-15T14:30:55Z

Made a few changes and merged!

woozyan added 2 commits March 15, 2018 17:00

Fix API tutorial for DyNet 2.0.3+

29ef2f4

Add more examples in basic Expression Manipulation

1ad8ded

xunzhang changed the title ~~Dynet api tutorial~~ [WIP] Dynet api tutorial Mar 16, 2018

woozyan added 2 commits March 16, 2018 23:32

add Expression init, more loss func and math func

d3d8edc

Add more loss, norm and examples

f9ff2b3

xunzhang reviewed Mar 19, 2018

View reviewed changes

woozyan mentioned this pull request Mar 20, 2018

batch_size in Expressions #1316

Closed

woozyan changed the title ~~[WIP] Dynet api tutorial~~ Dynet api tutorial Mar 22, 2018

xunzhang added 4 commits April 8, 2018 23:03

fix several places

5c43f36

another fix

0132186

update

81693b1

restart and clear output

ec0d1f4

xunzhang approved these changes Apr 9, 2018

View reviewed changes

pmichel31415 mentioned this pull request Apr 17, 2018

Examples on master not working with latest release #1349

Open

neubig requested changes Apr 20, 2018

View reviewed changes

neubig merged commit ec0d1f4 into clab:master Jun 15, 2018

Conversation

woozyan commented Mar 16, 2018 • edited by xunzhang Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

woozyan commented Mar 16, 2018

Uh oh!

xunzhang commented Mar 16, 2018

Uh oh!

pmichel31415 commented Mar 16, 2018

Uh oh!

xunzhang commented Mar 16, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xunzhang commented Mar 19, 2018

Uh oh!

neubig commented Apr 6, 2018

Uh oh!

xunzhang commented Apr 6, 2018

Uh oh!

xunzhang left a comment

Choose a reason for hiding this comment

Uh oh!

xunzhang commented Apr 9, 2018

Uh oh!

neubig left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

neubig commented May 11, 2018

Uh oh!

xunzhang commented May 11, 2018

Uh oh!

woozyan commented May 11, 2018

Uh oh!

neubig commented Jun 11, 2018

Uh oh!

xunzhang commented Jun 11, 2018

Uh oh!

neubig commented Jun 15, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

woozyan commented Mar 16, 2018 •

edited by xunzhang

Loading

xunzhang commented Mar 16, 2018 •

edited

Loading