
How to build your own neural network from scratch in Python - vikingo9
https://towardsdatascience.com/how-to-build-your-own-neural-network-from-scratch-in-python-68998a08e4f6
======
a_bonobo
Too often you see articles like this and they start with

$ import a_whole_bunch_of_stuff

Good to see that this is not the case here :)

The fast.ai course has a similar exercise in the beginning, but you'll still
import the weights from somewhere else.

Their fast.ai v1 library has a very short implementation too (loading the
MINIST example dataset and then using Resnet18):

    
    
        from fastai import *  
        from fastai.data import *  
        untar_data(MNIST_PATH)  
        data = image_data_from_folder(MNIST_PATH)  
        learn = ConvLearner(data, tvm.resnet18, metrics=accuracy)  
        learn.fit(1)
    

Done!

Source: [http://docs.fast.ai/](http://docs.fast.ai/)

~~~
sprobertson
How is your example not "$ import a_whole_bunch_of_stuff"?

~~~
ScoutOrgo
This notebook from fast.ai is more in line with what op mentioned. It starts
from nothing but Python and slowly swaps out chunks of code with
functions/classes from Torch/fastai:

[https://github.com/fastai/fastai/tree/master/courses/dl1](https://github.com/fastai/fastai/tree/master/courses/dl1)

------
maurits
Almost obligatory, but Stanford cs231n [1] lets you implement a complete
(deep) neural network from scratch [2] before venturing into pytorch/tf. Its
super fun.

[1] [http://cs231n.stanford.edu/](http://cs231n.stanford.edu/)

[2]
[http://cs231n.github.io/assignments2017/assignment1/](http://cs231n.github.io/assignments2017/assignment1/)

------
drej
I can highly recommend Joel Grus’ live coding video. He creates a deep
learning library only using numpy in an hour and it’s really fun to watch it
all come together. [https://youtu.be/o64FV-ez6Gw](https://youtu.be/o64FV-
ez6Gw)

~~~
lunchladydoris
That was really good! Thanks for the link.

------
tomglynch
Towards Data Science is definitely one of the best Medium Publications. I hope
they don't begin to monetise with Medium's member-only content. It's such a
drawback to Medium.

------
vesche
Nah, let's make that 10 lines:
[https://gist.github.com/vesche/72dafed33d614710f03f1b75cff1c...](https://gist.github.com/vesche/72dafed33d614710f03f1b75cff1c807)

I got a beer for anyone who can golf it under 5 lines

~~~
applecrazy
Naive solution (1 line):

[https://gist.github.com/applecrazy/deda2fac6e83c07b93e001731...](https://gist.github.com/applecrazy/deda2fac6e83c07b93e0017314de6cce)

Edit: I literally took newlines, converted to \n in a string, and then
exec()ed the whole thing. Here's a repl of it working:
[https://repl.it/@applecrazy/Code-Golfing-a-Neural-
Net](https://repl.it/@applecrazy/Code-Golfing-a-Neural-Net)

~~~
vesche
Oh dear, I got played.

PM address if you want the beer, you sly dog. Obligatory, I'm a normal dude:
[http://vesche.github.io/](http://vesche.github.io/)

Edit: Wait, you can't PM on here... PM on reddit /u/vesche

~~~
applecrazy
Well, I don't drink (see my bio), but thanks for the offer. :P

------
plg
Honest question: what are the reasons to code this up using OOP, creating a
neural network object plus methods, instead of data structures and functions
that operate on them?

If it’s just personal preference, I’m fine with that, I’m not trying to start
a flame war.

~~~
claytonjy
Python, both the language and community, are _very_ strong proponents of OOP.
While you _can_ do a lot of more functional stuff, esp. w/ functools, the
community at large tends to discourage that. "Never use map/filter" is a
weirdly common phrase among pythonistas. So this, like most python-driven
examples, is doing things in a _pythonic_ way.

Coming from the R side, I tend to prefer structures & functions as well, but
if I tried to write Python that way I'd be wary about showing that code to
anyone more entrenched in the Pythonic way of thinking.

~~~
jononor
The never use map/filter is in favor of list comprehensions, which is not
really a OOP construct.

But sure, it's a conventional thing. And for a long time the built-in
alternatives to a class for such a datastructure tuples and dicts, neither
which are very nice for functions to operate on (dict values have to be
addressed with d['key'] instead of d.key). With a class and method there is
also no doubt as to what the function operates on, which is convenient when
there type hints and IDE support is missing. This is changing since Python 3.5
and type checking tools like MyPy.

Since Python 3.7 there are also data classes, a primitive for classes which
just hold values. [https://www.linuxjournal.com/content/introducing-
python-37s-...](https://www.linuxjournal.com/content/introducing-
python-37s-dataclasses) But it will take a while before programming
conventions change.

------
derekmcloughlin
I found this very useful:

[https://makeyourownneuralnetwork.blogspot.com/](https://makeyourownneuralnetwork.blogspot.com/)
[https://www.amazon.com/dp/1530826608/ref=cm_sw_r_tw_dp_U_x_x...](https://www.amazon.com/dp/1530826608/ref=cm_sw_r_tw_dp_U_x_xmXTBbB5R5A2R)

First part goes into what a NN is and how backprop works, second part is an
implementation in Python.

------
bluemania
Thanks for submitting, this is going to come in handy for my education! I have
an assignment next week which requires creating a neural network, then
substituting various optimizers in place of backpropagation and comparing
performance over iterations. Have found that simple numpy based NN's are
easier to examine and connect the changes with the theory, this guide looks to
help further with this understanding!

~~~
dnautics
That sounds really fun! Do you get to pick optimizers? Obviously direct
gradient is a thing (calculate the partial for every parameter) but also
particle swarm optimization might be cool to profile.

------
peter303
I suggest a remarkably useful device: Four circular disks when attached to a
cart make it easy to move the cart from location to another. Furthermore you
could put goods in that cart and move them too. A lot easier than carrying the
goods on you back or an animal. A brief search of the internet finds this is a
new idea.

~~~
qbrass
What do the disks do? A cart already moves pretty easily on it's wheels, so
how much improvement have you made by attaching these disks to it?

------
amasad
Very cool! Here's the full version if you want fo run it/play with it

[https://repl.it/@turbio/neural-network](https://repl.it/@turbio/neural-
network)

------
starpilot
Another thing to try is calculating backpropagation by hand, on paper, with a
small NN. This is what my mentor said his final exam on NN in college
involved.

~~~
anonytrary
I would have thought calculus 1 was a prerequisite for that course.

------
wheresvic1
If you're looking for a similar from scratch tutorial in Java, check this one
out: [https://smalldata.tech/blog/2016/05/03/building-a-simple-
neu...](https://smalldata.tech/blog/2016/05/03/building-a-simple-neural-net-
in-java)

------
master_yoda_1
This is not as glorified as it sound. 100 of thousands of students around the
world build neural net from scratch as their homework. People please move on,
spend your time somewhere else which is more productive. I coded neural
network using c++ in 2009 as homework. But I never though about showing this
to people.

~~~
tobr
> People please move on

Strange reaction to a “beginner’s guide”, how can someone move on before they
learn the basics?

------
deytempo
Programming != math

