SkalskiP / ILearnDeepLearning.py
SkalskiP
/
ILearnDeepLearning.py
51
742281
Code
Issues 0
Pull requests 0 Actions
Projects 0
Wiki
Security 0
Insights
This repository contains small projects related to Neural Networks and Deep Learning in general. Subject are closely linekd with articles I publish on Medium. I encourage you both to read as well as to check how the code works in the action. https://medium.com/@piotr.skalski92
neural-network
deep-learning
visualizations
numpy
deep-learning-tutorial
computer-vision
200 commits
1 branch
0 packages
0 releases
4 contributors
MIT
Latest commit
@SkalskiP
SkalskiP plots up
Latest commit
60b2e5b
10 days ago
Files
Type Name Latest commit message Commit time
01_mysteries_of_neural_networks plots up 10 days ago
02_data_science_toolkit update last month
.gitignore explaining image classifiers initial commit tada 3 months ago
LICENSE.md Rename LICENSE to LICENSE.md 2 years ago
README.md Update README.md 3 months ago
logo.gif logo 2 years ago
requirements.txt Bump bleach from 2.1.3 to 3.1.4 (#25) 25 days ago
README.md
ILearnDeepLearning.py
NumPy NN animation
Description
People say that nothing develops and teaches you like getting your hands dirty. This repository contains small projects mostly related to Deep Learning but also Data Science in general. Subjects are closely linekd with articles I publish on Medium and are intended to complement those blog posts. For me it is a way to document my learning process, but also to help others understand neural network related issues. I hope that the content of the repository will turn out to be interesting and, above all, useful. I encourage you both to read my posts as well as to check how the code works in the action.
Hit the ground running
# clone repository
git clone https://github.com/SkalskiP/ILearnDeepLearning.py.git
# navigate to main directory
cd ILearnDeepLearning.py
# set up and activate python environment
apt-get install python3-venv
python3 -m venv .env
source .env/bin/activate
# install all required packages
pip install -r requirements.txt
Deep Dive into Math Behind Deep Networks
Medium articule – Source code
This project is mainly focused on visualizing quite complex issues related to gradient descent, activation functions and visualization of classification boundaries while teaching the model. It is a code that complements the issues described in more detail in the article. Here are some of the visualizations that have been created.
Keras model frames Keras class boundries
Figure 1. A classification boundaries graph created in every iteration of the Keras model.
Finally, the frames were combined to create an animation.
Gradient descent
Figure 2. Visualization of the gradient descent.
Let’s code a Neural Network in plain NumPy
Medium articule – Source code
After a theoretical introduction, the time has come for practical implementation of the neural network using NumPy. In this notebook you will find full source code and a comparison of the performance of the basic implementation with the model created with Keras. You can find a wider commentary to understand the order and meaning of performed functions in a related article.
NumPy NN animation
Figure 3. Visualisation of the classification boundaries achieved with simple NumPy model
Preventing Deep Neural Network from Overfitting
Medium articule – Source code
This time I focused on the analysis of the reasons for overfitting and ways to prevent it. I made simulations of neural network regulation for different lambda coefficients, analyzing the change of values in the weight matrix. Take a look at the visualizations that were created in the process.
Change of accuracy
Figure 4. Classification boundaries created by: top right corner - linear regression;
bottom left corner - neural network; bottom right corner - neural network with regularisation
Change of accuracy
Figure 5. Change of accuracy values in subsequent epochs during neural network learning.
How to train Neural Network faster with optimizers?
Medium articule – Source code
As I worked on the last article, I had the opportunity to create my own neural network using only Numpy. It was a very challenging task, but at the same time it significantly broadened my understanding of the processes that take place inside the NN. Among others, this experience made me truly realize how many factors influence neural net’s performance. Selected architecture,proper hyperparameter values or even correct initiation of parameters, are just some of those things… This time however, we will focus on the decision that has a huge impact on learning process speed, as well as the accuracy of obtained predictions - the choice of the optimization strategy.
Change of accuracy
Figure 6. Examples of points which are a problem for optimization algorithms.
Change of accuracy
Figure 7. Optimizers comparison.
Simple Method of Creating Animated Graphs
Medium articule – Source code
Both in my articles and projects I try to create interesting visualizations, which very often allow me to communicate my ideas much more effectively. I decided to create a short tutorial to show you how to easily create animated visualizations using Matplotlib. I also encourage you to read my post where I described, among other things, how to create a visualization of neural network learning process.
Change of accuracy
Figure 8. Lorenz Attractor created using the Matplotlib animation API.
Gentle Dive into Math Behind Convolutional Neural Networks
Medium articule – Source code
In this post on Medium I focused on the theoretical issues related to CNNs. It is a preparation for the upcoming mini project, which aims to create my own, simple implementation of this type of the Neural Network. As a result, this section of the repository is quite narrow and includes mainly simple visualizations of the effects of a convolution with a selected filter.
Convolution
Figure 9. Convolutionary effect with selected filters.
Chess, rolls or basketball? Let’s create a custom object detection model
Medium articule – Source code
My posts on the Medium are usually very theoretical – I tend to analyse and describe the algorithms that define how Neural Networks work. This time, however, I decided to break this trend and show my readers how easy it is to train your own YOLO model, capable of detecting any objects we choose. In order to achieve this goal, we will need help from a very useful and easy-to-use implementation of YOLO. In short, not much coding, but a huge effect.
Convolution
Figure 10. Detection of players moving around the basketball court,
based on YouTube-8M dataset.
Knowing What and Why? - Explaining Image Classifier Predictions
Medium articule – Source code
As we implement highly responsible Computer Vision systems, it is becoming progressively clear that we must provide not only predictions but also explanations, as to what influenced its decision. In this post, I compared and benchmarked the most commonly used libraries for explaining the model predictions in the field of Image Classification - Eli5, LIME, and SHAP. I investigated the algorithms that they leverage, as well as compared the efficiency and quality of the provided explanations.
Explaining predictions
Figure 11. Comparison of explanations provided by ELI5, LIME and SHAP
License
This project is licensed under the MIT License – see the LICENSE.md file for details
Interesting materials and ideas
This is a place where I collect links to interesting articles and papers, which I hope will become the basis for my next projects in the future.
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Sequence to Sequence Learning with Neural Networks
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
BLEU: a Method for Automatic Evaluation of Machine Translation
Neural Machine Translation by Jointly Learning to Align and Translate
A (Long) Peek into Reinforcement Learning
Why Momentum Really Works
Improving the way neural networks learn
Classification and Loss Evaluation – Softmax and Cross Entropy Loss
© 2020 GitHub, Inc.
Terms
Privacy
Security
Status
Help
Contact GitHub
Pricing
API
Training
Blog
About
