
begin to improve the spatial resolution of HSI by means of
SR reconstruction methods.
Up to now, many HSI SR methods have been proposed
to improve the resolutio n of HSIs. Some fusion methods
have been proposed for obtaining the HR HSI by com-
bining a LR HSI with a HR panchromatic image (covering
a large spectral window), such as linear transformations
(e.g., intensity-hue-saturation (IHS) transform [6], princi-
pal component analysis, wavelet transform [7–9]), unmix-
ing-based [10–12], and joint filtering [13]. Those
approaches, originally developed by the community of
remote sensing, have been known as pansharpening and
especially suitable for the case where the spectral-resolu-
tion difference between two input images is relatively
small. With the development of sparse representation the-
ory [14], some sparsity-based methods have also been
proposed for HSI SR. Zhao et al. [15] proposed sparse
representation and spectral regularization algorithm that
uses the sparse prior to enhance the spatial resolution.
Simoes et al. [16] used a regularized form based on vector
total variation to fuse LR HSI wi th HR multispectral
images to get HR HSI. Dong et al. [17] proposed a non-
negative structure sparse representation model that trans-
forms the HSI SR into a joint estimate of spectral basis and
sparse coefficients. However, most of these methods
tackled the HSI SR problem as an image fusion problem
using an auxiliary HR image. In the reality, it is very dif-
ficult to obtain a couple of HR panchromatic image and
HSI about the same scene with completely registration,
which makes this kind of method not so practical.
Recently, as convol ution neural network (CNN) can
extract the high-level features and explore the contextual
information, it has been widely applied in man y computer
vision fields, such as face reco gnition [18], visual recog-
nition [19], natural image super-resolution [20], and so on.
In the last two years, CNN has also been applied in the HSI
SR. Yuan et al. [21] transferred the CNN with three con-
volution layers in [22] to learn the mapping relationship
between LR and HR HSI images. But this method did not
consider the spectral information preservation and the
difference between HSI and RGB images. For this, Li et al.
[5] p roposed an HSI SR method by combining a deep
spectral difference convolutional neural network (SDCNN)
with a spatial constraint (SCT) strategy, denoted by SCT-
SDCNN method. It still uses the CNN as in paper [21] with
three convolution layers but to learn the spectra l difference
mapping between the LR HSI and HR HSI . Furthermore, it
applies SCT strategy to constrain the LR HSI generat ed by
the reconstructed HR HSI spatially close to the original LR
HSI. Subsequently, people begin to consider constructing
more complex deep networks for HSI SR inspired the work
done for natural image SR. For example, He et al. [23]
proposed an HSI SR method inspired by a deep Laplacian
pyramid network to enhance the spatial resolution with the
spectral information preserved . Then, a nonnegative dic-
tionary learning method is proposed for spectral informa-
tion reconstruction. These deep-network-based HSI SR
methods have good reconstruction effect because of the
outstanding performance of deep networks. What is more,
these methods do not need the auxiliary panchromatic
image and multispectral image of the same scene.
Generally speaking, for given training data, before the
network is over-fitted, the deeper the network is, the better
the reconstruction effect is. But increasing the depth of the
network will make the number of network parameters
(weight and bias) explode quickly. For example, even
highway network [24] with about 18 convolution layers for
the natural image SR, its number of parameters is about
1,000,000. Since HSI consists of many 2D images in dif-
ferent bands, which implies that HSI SR needs much more
model parameters than the natural image does. Therefore, it
will bring some troubles when it is applied in mobile ter-
minal devices. The reason is that when the depth of the
network increases, it needs not only more memory for
computation but also larger storage space. Of course, the
ordinary mobile terminal devices cannot satisfy the com-
puting requirement. Therefore, it is instructive to design a
deep network for HSI SR with less weights. Huang et al.
[25] proposed the idea of dense connection in DenseNet to
strengthen the feature propagation and encourage the fea-
ture reuse . It makes DenseNet use only a third of param-
eters in ResNet [
26] with the same effect. But DenseNet is
used to solve the classification problem, not even for nat-
ural image SR.
In this paper, we want to propose a compact deep net-
work for HSI SR with less model parameters. We adopt the
idea of dense connection in DenseNet [25] to construct a
dense block for the deep network. But in order to further
reduce the network parameters in DenseNet while keeping
the approximate performance, we fuse the recursion idea
[27] on dense connection to design a more compact deep
network for HSI SR. We call the network recursive densely
connected neural network (RDCNN). The proposed
RDCNN uses the recursion to share the weights and biases
in the dense bloc k for reducing the number of model
parameters. Furthermore, with the SCT strategy, the
reconstructed HR HSI by RDCNN can be improved fur-
thermore. We name this HSI SR method as recursive
densely convolutional neural network with a spatial con-
straint strategy (SCT-RDCNN). The proposed SCT-
RDCNN can learn the mapping relation between the LR
HSI and HR HSI directly instead of the band difference in
SCT-SDCNN. The way of dense connection in our pro-
posed RDCNN can not only extract the high-level featur e
and allevi ate the problem of gradient vanishing and
exploding, but also use less weights than the network
Neural Computing and Applications
123