Huawei H13-311_V3.5 Practice Test - Questions Answers, Page 4

In a fully-connected layer, the number of parameters is calculated by multiplying the number of input features by the number of neurons in the hidden layer. For an image of resolution 100100=10,000100 \times 100 = 10,000100100=10,000 pixels and a hidden layer of 1,000 neurons, the total number of parameters is 10,0001,000=1,000,00010,000 \times 1,000 = 1,000,00010,0001,000=1,000,000.

The global gradient descent algorithm evaluates the gradient over the entire dataset before each update, leading to accurate but slow convergence, especially for large datasets. In contrast, stochastic gradient descent updates the model parameters more frequently, which allows for faster convergence but with noisier updates. While batch gradient descent updates the parameters based on smaller batches of data, none of these algorithms can fully guarantee finding the global minimum in non-convex problems, where local minima may exist.

Activation functions like Sigmoid, tanh, and softsign suffer from the vanishing gradient problem when used in deep networks. This happens because, in these functions, gradients become very small as the input moves away from the origin (either positively or negatively). As a result, the weights of the earlier layers in the network receive very small updates, hindering the learning process in deep networks. This is one reason why activation functions like ReLU, which avoid this issue, are often preferred in deep learning.

Both single-layer perceptrons and logistic regression are linear classifiers, meaning they are capable of separating data that is linearly separable. However, they cannot effectively model non-linear relationships in the data. For more complex, non-linearly separable data, multi-layer neural networks or other non-linear classifiers are required.