Description
tensorFlow version (use command below): tensorflow 2.2
Python version: 3.7.9
CUDA/cuDNN version: 10.1
I'm traing a resnet50 model,but the moving_mean and moving_variance in batchnormalization going to very big after 20 epoch ,and when i use this model to predit the mean and variance of the same layer is different with the moving_mean and moving_variance.
bn_conv1 is the first batchnormalization. bn5c_branch2b (BatchNormalization) in the last identity_block:
bn_conv1 (BatchNormalization)
bn5c_branch2b (BatchNormalization):
conv1 (Conv2D) (None, 402, 6, 32) 224 reshape[0][0]
bn_conv1 (BatchNormalization) (None, 402, 6, 32) 128 conv1[0][0]
leaky_re_lu (LeakyReLU) (None, 402, 6, 32) 0 bn_conv1[0][0]
max_pooling2d (MaxPooling2D) (None, 200, 6, 32) 0 leaky_re_lu[0][0]
........
........
res5c_branch2b (Conv2D) (None, 13, 6, 128) 147584 leaky_re_lu_46[0][0]
bn5c_branch2b (BatchNormalizati (None, 13, 6, 128) 512 res5c_branch2b[0][0]
leaky_re_lu_47 (LeakyReLU) (None, 13, 6, 128) 0 bn5c_branch2b[0][0]
res5c_branch2c (Conv2D) (None, 13, 6, 1024) 132096 leaky_re_lu_47[0][0]
bn5c_branch2c (BatchNormalizati (None, 13, 6, 1024) 4096 res5c_branch2c[0][0]
add_15 (Add) (None, 13, 6, 1024) 0 bn5c_branch2c[0][0]
leaky_re_lu_45[0][0]
leaky_re_lu_48 (LeakyReLU) (None, 13, 6, 1024) 0 add_15[0][0]
max_pooling2d_1 (MaxPooling2D) (None, 4, 2, 1024) 0 leaky_re_lu_48[0][0]
flatten (Flatten) (None, 8192) 0 max_pooling2d_1[0][0]