Repository navigation
Integrate with TensorFlow.jl #29
Description
Activity
I'm confused as to what this would even mean.
I think I had the idea that TensorFlow probably needs some data input, and that it would be nice if it could consume an iterable table. But, I have no idea how TensorFlow works, so this might indeed be a nonsense idea :)
The main way to get data into TensorFlow is
passing it in viarun. where you loadup your data in julia, then pass it as an argument to the tensorflow run.
Which works just fine I assume already.
Egrun(sess, Ycancerrisk, Dict(Xage=df[:age], Xoccupation=df[:occupation]).
This is the only way I've ever loaded data into TensorFlow myself.The other way is using various things like readers
so that the tensorflow C backend loads the data directly.
I'm not sure how queues fit into the picture.
It might be that there is a place for IterableTables.jl in integrating with queues.I guess one thing that would work is that one can pass a table directly as the third argument, instead of the
Dict, and then it will use the data from the columns by name. For example with your line it would berun(sess, Ycancerrisk, df)And the
dfvariable could be any iterable table that has a columnXageandXoccupation. Would that be useful?It would also mean that you could pass in a Query.jl result directly, without materializing it first:
q = @from i in df begin @select {Xage=i.age, XOccupation=i.occupation} end run(sess, Ycancerrisk, q)
We could also enable the piping syntax that I showed at juliacon, so that things like this would work:
df |> run(sess, Ycancerrisk) df |> @query(i, begin @select {Xage=i.age, XOccupation=i.occupation} end) |> run(sess, Ycancerrisk)
I have no idea how useful any of this would be :)
And the df variable could be any iterable table that has a column Xageand Xoccupation. Would that be useful?
Maybe,
It would basically come for free, if thefeed_dict(egDict(Xage=df[:age], Xoccupation=df[:occupation])),
was changed fromAssociative{Tensor, Any}to be allowed to beAssociative{Symbol, Any}.
Which requiresSymbols to be resolved to theTensornodes they represent.
Which can be done right now, generally.
I have had some general thoughts in this direction about revampingrun.
(Probably also with a macro form)It would also mean that you could pass in a Query.jl result directly, without materializing it first:
If you mean delaying computation via lazy evaluation then it is not possible (without substantial additional magic).
Not too deep behindrunis the datacollected so that it can be sent over FFI to the tensorflow C/C++ backend.Ah, I had not looked carefully enough at the
Dictthere. Another option would be to pass in an iterable table, and the day a list of pairs that map tensor objects to symbols that refer to the columns. Not a huge improvement, but it would save a few characters if typing, I guess.
If possible.