58 Resources To Help Get Started With Deep Learning ( In TF )

Fifty-eight blogs/videos to enter Deep Learning with TensorFlow ( along with 8 memes to keep you on track )

Photo by Pauline Loroy on Unsplash

Deep Learning could be super fascinating if you’re already in love with other ML algorithms. DL is all about huge neural networks, with numerous parameters, different layers which perform specific tasks.

This checklist will provide you with a smooth start ( and also a safer start ) with Deep Learning in TensorFlow.

Warning: This story uses some dreaded terminology. Readers who have jumped directly into DL, thereby skipping basic ML algos, are advised.

Without wasting time, let’s get started.

? The 30,000 feet overview ( Contents )

? The Basics

Mathematics of backpropagation ( bonus, refresher ),

? TensorFlow Essentials

? Entering the DL World

⚔️ Training the Model

??‍? More from the Author

? Contribute to this story

? The Basics

? Understanding Neural Networks

Artificial Neural Networks are systems inspired by biological neurons and are used to solve complex problems, as they have the unique ability to approximate functions, which map the inputs to their labels.

By Tony Yiu,

Understanding Neural Networks

And 3Blue1Brown is a must,

☠️ Your attention here, please. The following topic which DL folks call “backpropagation” is a serial killer for many ML beginners. It roams around with creepy weapons like partial derivatives, chain rule, so beginners with weak mathematics are advised.

ML: Theory vs. Practice
Big Data Jobs

? Backpropagation

Backpropagation is used to optimize the parameters of ML models, which aims to find a minimum for the given cost function. Having a strong understanding of how backpropagation works under the hood, is essential in deep learning.

This 3Blue1Brown playlist might be helpful,

? TensorFlow Essentials

Although the TensorFlow website is enough for a beginner to get started, I would like to focus on some important topics.

? Dataset

This is an important class, which helps us load huge datasets easily and feed them to our NNs. The Dataset class has a bunch of useful methods. Using the dataset.map() method along with various methods in tf.image module, we can perform a number of operations on images from our data.

? TensorBoard

TensorBoard will help you visualize the values of various metrics used during the training of our model. It could be the accuracy of our model or the MAE ( Mean Absolute Error ) in case of a regression problem.

The official docs,

TensorBoard | TensorFlow

TensorBoard.dev can help you share the results of your experiments easily,

TensorBoard.dev – Upload and Share ML Experiments for Free

Some other helpful resources,

? Entering the DL World

A good analogy to understand the working of NN.

? Activation Functions

Activation functions introduce non-linearity in our NN, so as to approximate the function which would map our input data with the labels. Every activation function has its own pros and cons, so the choice will always depend on your use case.

Also, have a look at the tf.keras.activations module, to see the available activation functions in TensorFlow,

Module: tf.keras.activations | TensorFlow Core v2.4.1

Trending AI Articles:

1. Why Corporate AI projects fail?

2. How AI Will Power the Next Wave of Healthcare Innovation?

3. Machine Learning by Using Regression Model

4. Top Data Science Platforms in 2021 Other than Kaggle

? Dense Layers

Dense layers are quite important in every possible NN. You’ll find them in every architecture whether it is for text classification, image classification or GANs. Here’s a good read, by Hunter Heidenreich, about tf.keras.layers.Dense to have a better understanding,

Understanding Keras — Dense Layers

? Batch Normalization

Batch normalization is one of the most efficient techniques to accelerate the training of NN, by tackling internal covariate shift. Some good blogs you may read,

From Richmond Alake,

Batch Normalization In Neural Networks Explained (Algorithm Breakdown)

Andrew Ng’s explanation on Batch Normalization,

And lastly, make sure you explore the tf.keras.layers.BatchNormalization class,

tf.keras.layers.BatchNormalization | TensorFlow Core v2.4.1

? Dropout

Dropout is widely used in NNs, as it fights efficiently against overfitting. It randomly sets activations to 0, given a dropout_rate which is the probability of activation being set to 0.

Its always better to start with a question in mind,

By Amar Budhiraja,

Learning Less to Learn Better — Dropout in (Deep) Machine learning

Understanding the math behind Dropout, in a simple and lucid manner,

Simplified Math behind Dropout in Deep Learning

Make sure you explore the official TF docs for tf.keras.layers.Dropout ,

tf.keras.layers.Dropout | TensorFlow Core v2.4.1

? Weight Initialization Techniques

Considering Dense layers, the parameters W and b i.e the weights and the biases, are randomly initialized. They are then optimized by Gradient Descent ( backpropagation ). But there are some smart ways to initialize these parameters so that the loss function could quickly each the minima. Some good reads are,

By Saurabh Yadav,

The story below will give you a glimpse of how and where should we use different weight initialization techniques, by James Dellinger,

Weight Initialization in Neural Networks: A Journey From the Basics to Kaiming

Lastly, explore the tf.keras.initializers module,

Keras documentation: Layer weight initializers

? Training the Model

? Optimizers ( Variants of Gradient Descent )

A number of different optimizers are available in the tf.keras.optimizers module. Each optimizer is a variant or improvement to the gradient descent algorithm.

By Imad Dabbura,

By Raimi Karim,

10 Gradient Descent Optimisation Algorithms

? Loss Functions

Cost functions ( synonymously called loss functions ) penalize the model for incorrect predictions. It measures how good the model is, and decides how much improvement is needed. Different loss functions have different use-cases, which are detailed thoroughly in the stories below,

A great video from Stanford,

? Learning Rate Schedule

By practice, the learning rate of a NN model should be decreased over time. As the value of the loss function decreases, or as the loss function is close to minima, we take smaller steps. Note, the learning rate decides the step size of gradient descent.

Make sure you explore the Keras docs,

Keras documentation: LearningRateScheduler

? Early Stopping

Early stopping is a technique wherein we stop the training of our model when a given metric stops improving. So, we stop the training of our model, before it overfits, thereby avoiding any excessive training which can worsen the results.

By Upendra Vijay,

Early Stopping to avoid overfitting in neural network- Keras

? Batch Size ( and Epochs vs. Iterations vs. Batch Size )

Batch size is the number of samples present in a mini-batch. Each batch is sent through the NN and the errors are averaged across the samples. So, for each batch, the parameters of the NN are optimized. Also, see Mini-Batch Gradient Descent.

By Kevin Shen,

Effect of batch size on training dynamics

This could probably clear a common confusion among beginners,


Epoch vs Batch Size vs Iterations

? Metrics

Metrics are functions whose value is evaluated to see how good a model is. The primary difference between a metric and a cost function is that the cost function is used to optimize the parameters of the model in order to reach the minima. A metric could be calculated at each epoch or step in order to keep track of the model’s efficiency.

By Shervin Minaee,

20 Popular Machine Learning Metrics. Part 1: Classification & Regression Evaluation Metrics

By Aditya Mishra,

Sometimes, we may require some metrics which aren’t available in the tf.keras.metrics module. In this case, a custom metric should be implemented, as described in this blog,

Custom Loss and Custom Metrics Using Keras Sequential Model API

? Regularization

Regularization consists of those techniques which help our model prevent overfitting and thereby generalize itself better.

By Richmond Alake,

Excellent videos from Andrew Ng,

✌️ More from the Author

✌️ Thanks

A bonus tip, you might try!

A super big read, right? Did you find some other blog/video/book useful, which could be super-cool for others as well? Well, you’re at the right place! Send an email at equipintelligence@gmail.com to showcase your resource in this story ( the credits would of course go with you! ).

Goodbye and a happy Deep Learning journey!

Don’t forget to give us your ? !

58 Resources To Help Get Started With Deep Learning ( In TF ) was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.

Via https://becominghuman.ai/58-resources-to-help-get-started-with-deep-learning-in-tf-9f29ba585e0f?source=rss—-5e5bef33608a—4

source https://365datascience.weebly.com/the-best-data-science-blog-2020/58-resources-to-help-get-started-with-deep-learning-in-tf

Published by 365Data Science

365 Data Science is an online educational career website that offers the incredible opportunity to find your way into the data science world no matter your previous knowledge and experience. We have prepared numerous courses that suit the needs of aspiring BI analysts, Data analysts and Data scientists. We at 365 Data Science are committed educators who believe that curiosity should not be hindered by inability to access good learning resources. This is why we focus all our efforts on creating high-quality educational content which anyone can access online.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Create your website with WordPress.com
Get started
%d bloggers like this: