Keras
https://keras.io/
Simple. Flexible. Powerful.
Deep learning for humans.
Keras is an API designed for human beings, not machines.
Keras follows best practices for reducing cognitive load: it offers consistent & simple APIs, it minimizes the number of user actions required for common use cases, and it provides clear & actionable error messages.
It also has extensive documentation and developer guides.
mnist
https://keras.io/api/datasets/mnist/
load_datafunctiontf.keras.datasets.mnist.load_data(path="mnist.npz")Loads the MNIST dataset.
This is a dataset of 60,000 28x28 grayscale images of the 10 digits, along with a test set of 10,000 images. More info can be found at the MNIST homepage.
Arguments
- path: path where to cache the dataset locally (relative to
~/.keras/datasets).Returns
- Tuple of Numpy arrays:
(x_train, y_train), (x_test, y_test).- x_train, x_test: uint8 arrays of grayscale image data with shapes (num_samples, 28, 28).
- y_train, y_test: uint8 arrays of digit labels (integers in range 0-9) with shapes (num_samples,).
License: Yann LeCun and Corinna Cortes hold the copyright of MNIST dataset, which is a derivative work from original NIST datasets. MNIST dataset is made available under the terms of the Creative Commons Attribution-Share Alike 3.0 license.
http://yann.lecun.com/exdb/mnist/
The MNIST database of handwritten digits, available from this page, has a training set of 60,000 examples, and a test set of 10,000 examples. It is a subset of a larger set available from NIST. The digits have been size-normalized and centered in a fixed-size image.
It is a good database for people who want to try learning techniques and pattern recognition methods on real-world data while spending minimal efforts on preprocessing and formatting.
安装
https://www.tensorflow.org/install/pip
pip install --upgrade tensorflow
Simple MNIST convnet
https://keras.io/examples/vision/mnist_convnet/
https://github.com/keras-team/keras-io/blob/master/examples/vision/mnist_convnet.py
""" Title: Simple MNIST convnet Author: [fchollet](https://twitter.com/fchollet) Date created: 2015/06/19 Last modified: 2020/04/21 Description: A simple convnet that achieves ~99% test accuracy on MNIST. """ """ ## Setup """ import numpy as np from tensorflow import keras from tensorflow.keras import layers """ ## Prepare the data """ # Model / data parameters num_classes = 10 input_shape = (28, 28, 1) # the data, split between train and test sets (x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data() # Scale images to the [0, 1] range x_train = x_train.astype("float32") / 255 x_test = x_test.astype("float32") / 255 # Make sure images have shape (28, 28, 1) x_train = np.expand_dims(x_train, -1) x_test = np.expand_dims(x_test, -1) print("x_train shape:", x_train.shape) print(x_train.shape[0], "train samples") print(x_test.shape[0], "test samples") # convert class vectors to binary class matrices y_train = keras.utils.to_categorical(y_train, num_classes) y_test = keras.utils.to_categorical(y_test, num_classes) """ ## Build the model """ model = keras.Sequential( [ keras.Input(shape=input_shape), layers.Conv2D(32, kernel_size=(3, 3), activation="relu"), layers.MaxPooling2D(pool_size=(2, 2)), layers.Conv2D(64, kernel_size=(3, 3), activation="relu"), layers.MaxPooling2D(pool_size=(2, 2)), layers.Flatten(), layers.Dropout(0.5), layers.Dense(num_classes, activation="softmax"), ] ) model.summary() """ ## Train the model """ batch_size = 128 epochs = 15 model.compile(loss="categorical_crossentropy", optimizer="adam", metrics=["accuracy"]) model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, validation_split=0.1) """ ## Evaluate the trained model """ score = model.evaluate(x_test, y_test, verbose=0) print("Test loss:", score[0]) print("Test accuracy:", score[1])
output
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
11493376/11490434 [==============================] - 14s 1us/step
x_train shape: (60000, 28, 28, 1)
60000 train samples
10000 test samples
WARNING:tensorflow:From /root/.pyenv/versions/3.6.8/lib/python3.6/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 26, 26, 32) 320
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 13, 13, 32) 0
_________________________________________________________________
conv2d_1 (Conv2D) (None, 11, 11, 64) 18496
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 5, 5, 64) 0
_________________________________________________________________
flatten (Flatten) (None, 1600) 0
_________________________________________________________________
dropout (Dropout) (None, 1600) 0
_________________________________________________________________
dense (Dense) (None, 10) 16010
=================================================================
Total params: 34,826
Trainable params: 34,826
Non-trainable params: 0
_________________________________________________________________
Train on 54000 samples, validate on 6000 samples
2021-03-15 16:39:38.250130: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2021-03-15 16:39:38.260145: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2112000000 Hz
2021-03-15 16:39:38.262184: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7ffff254b980 executing computations on platform Host. Devices:
2021-03-15 16:39:38.262538: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2021-03-15 16:39:38.312421: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
Epoch 1/15
54000/54000 [==============================] - 28s 513us/sample - loss: 0.3700 - acc: 0.8849 - val_loss: 0.0806 - val_acc: 0.9788
Epoch 2/15
54000/54000 [==============================] - 24s 447us/sample - loss: 0.1105 - acc: 0.9656 - val_loss: 0.0541 - val_acc: 0.9845
Epoch 3/15
54000/54000 [==============================] - 26s 476us/sample - loss: 0.0840 - acc: 0.9746 - val_loss: 0.0457 - val_acc: 0.9873
Epoch 4/15
54000/54000 [==============================] - 24s 450us/sample - loss: 0.0727 - acc: 0.9784 - val_loss: 0.0431 - val_acc: 0.9878
Epoch 5/15
54000/54000 [==============================] - 31s 576us/sample - loss: 0.0636 - acc: 0.9804 - val_loss: 0.0392 - val_acc: 0.9883
Epoch 6/15
54000/54000 [==============================] - 24s 453us/sample - loss: 0.0573 - acc: 0.9824 - val_loss: 0.0344 - val_acc: 0.9905
Epoch 7/15
54000/54000 [==============================] - 26s 478us/sample - loss: 0.0526 - acc: 0.9834 - val_loss: 0.0324 - val_acc: 0.9913
Epoch 8/15
54000/54000 [==============================] - 24s 438us/sample - loss: 0.0492 - acc: 0.9845 - val_loss: 0.0339 - val_acc: 0.9905
Epoch 9/15
54000/54000 [==============================] - 23s 429us/sample - loss: 0.0452 - acc: 0.9855 - val_loss: 0.0311 - val_acc: 0.9905
Epoch 10/15
54000/54000 [==============================] - 23s 430us/sample - loss: 0.0432 - acc: 0.9864 - val_loss: 0.0299 - val_acc: 0.9913
Epoch 11/15
54000/54000 [==============================] - 23s 428us/sample - loss: 0.0412 - acc: 0.9867 - val_loss: 0.0291 - val_acc: 0.9922
Epoch 12/15
54000/54000 [==============================] - 23s 421us/sample - loss: 0.0394 - acc: 0.9875 - val_loss: 0.0306 - val_acc: 0.9902
Epoch 13/15
54000/54000 [==============================] - 23s 418us/sample - loss: 0.0376 - acc: 0.9880 - val_loss: 0.0268 - val_acc: 0.9917
Epoch 14/15
54000/54000 [==============================] - 22s 404us/sample - loss: 0.0347 - acc: 0.9889 - val_loss: 0.0296 - val_acc: 0.9912
Epoch 15/15
54000/54000 [==============================] - 22s 399us/sample - loss: 0.0360 - acc: 0.9884 - val_loss: 0.0269 - val_acc: 0.9917
Test loss: 0.02560416207953822
Test accuracy: 0.9908
Epoch vs Iteration
https://stackoverflow.com/questions/4752626/epoch-vs-iteration-when-training-neural-networks
618In the neural network terminology:
- one epoch = one forward pass and one backward pass of all the training examples
- batch size = the number of training examples in one forward/backward pass. The higher the batch size, the more memory space you'll need.
- number of iterations = number of passes, each pass using [batch size] number of examples. To be clear, one pass = one forward pass + one backward pass (we do not count the forward pass and backward pass as two different passes).
Example: if you have 1000 training examples, and your batch size is 500, then it will take 2 iterations to complete 1 epoch.
FYI: Tradeoff batch size vs. number of iterations to train a neural network