TesorFlow is one of the most well-known deep learning frameworks. This posting will describe the basic steps for training a model with Tensorflow. Briefly, the basic steps for how to train a model through TensorFlow are as follow.

  1. Load & pre-process the dataset
  2. Set a model
  3. Compile the model : set an optimizer, a loss function, & metrics
  4. Fit the model
  5. Evaluate the model

Full code can be found at the following github repository.

This post was written with reference to the following materials.

1. Load & pre-process the dataset

For the experiment to show the basic process of training a model with TensorFlow, we used the MNIST dataset. The given images are a grayscale, so we need a preprocessing process to scale the image data features from 0 to 1.


2. Set a model

In this step, we designed the structure of a model. The model in this experiment was relatively simple, so it could be easier to construct the model using tf.keras.Sequential. However, we use custumized model for the experiment. This is because a custumized model must be used when constructing a more complex model in the future, and it is necessary to become familiar with it. More specifically, in case of the model becomes complicated and the input features do not flow sequentially, the model must be constructed in a custum manner. Transformer model is a good example.


(optional) Set callbacks

checkpoint

We set checkpoint callback to record the training process of the model.


tensorboard

We set tensorboard callback to visualize the training process and strucutre of the model.


reduce learning rate

We set reduce-learning-rate callback for better training result of the model.


3. Compile the model

Set optimizer, loss function, & metrics

After constructing the model, you need to set the learning process by calling the compile method. Compile method has three important parameters; optimizer, loss, and metrics.

The optimizer sets up the training process. The loss sets the loss function to be minimized during the optimization process. The metrics are used to monitor training. These should be set differently depending on the type of feature data and the purpose of the model.

The model was compiled using the set optimizer, loss, and metrics.


4. Fit the model

We conducted training using the preprocessed dataset and the compiled model.


(Optional) Check the training process via tensorboard

Through the tensorboard callback set in the previous step, we visualized the training process and the model graph. Figure 1 shows the training process of the model.

Figure 1. Checking the training process with tensorboard.


(Optional) Load the best weights

Through the checkpoint callback set in the previous step, we loaded the weights showing the best performance.


5. Evaluate the model

As a final step, we evaluated the performance of the model using the test dataset.